Loading ...
Sorry, an error occurred while loading the content.

Spider sitemap

Expand Messages
  • Nick
    Can Xenu take a Google sitemap as an input and spider all pages listed and check for broken links (both in the sitemap and another level deep)? If not, this
    Message 1 of 4 , Apr 13, 2009
    • 0 Attachment
      Can Xenu take a Google sitemap as an input and spider all pages listed and check for broken links (both in the sitemap and another level deep)?

      If not, this would be a great feature. I run a few sites where if I let Xenu loose on the site it will take days to finish spidering. Giving it a list of URLs would be much simpler.

      Thanks
      Nick
    • Ignacio Seron
      One way to solve this is ( without asking Tilman for more development): 1. Download the target XML sitemap 2. Open it with Excel or similar. 3. Select the
      Message 2 of 4 , Apr 14, 2009
      • 0 Attachment
        One way to solve this is ( without asking Tilman for more development):

        1. Download the target XML sitemap
        2. Open it with Excel or similar.
        3. Select the valid URLs and copy them
        4. Create a file with the list of URLs, with one URL in each line.
        5. Go to XENU and go to File > Check URL list ( test)
        6. Open the file with URLs .
        7. ready. :-)
        It is not so complicated to to it.
        regards
        Ignacio



        2009/4/14 Nick
        <noodles@...>



        Can Xenu take a Google sitemap as an input and spider all pages listed and check for broken links (both in the sitemap and another level deep)?

        If not, this would be a great feature. I run a few sites where if I let Xenu loose on the site it will take days to finish spidering. Giving it a list of URLs would be much simpler.

        Thanks
        Nick


      • Thomas Fischer
        Hello! ... My understanding is that this won t work as intended. The File Check URL list ( test) is a multiple Check URL feature, which would spawn
        Message 3 of 4 , Apr 14, 2009
        • 0 Attachment
          Hello!

          > One way to solve this is ( without asking Tilman for more
          > development):
          >
          >
          >
          > 1. Download the target XML sitemap
          >
          > 2. Open it with Excel or similar.
          > 3. Select the valid URLs and copy them
          > 4. Create a file with the list of URLs, with one URL in each line.
          > 5. Go to XENU and go to File > Check URL list ( test)
          > 6. Open the file with URLs .
          > 7. ready. :-)
          >

          My understanding is that this won't work as intended.
          The "File > Check URL list ( test)" is a multiple "Check URL" feature, which
          would spawn separate crawlings for each URL, which is not the goal here.
          If you want to check a list of URLs, the only way seems to be to create
          links
          <A HREF="...">Anything</A>
          and store them in a file. Then use the "Check URL..." with the "Local
          File..." option. You must mark the "Check external links" checkbox, because
          all the URLs you give are external to the local file, and you might end up
          with some URLs checked that are more remote than you expected.
          By the way, I would use a text editor to extract the URLs from the sitemap
          and create the links; on Windows Notepad++ will do nicely, as will
          TextWrangler on a Mac.

          All the best
          Thomas
        • Tilman Hausherr
          Thomas is right :-) Honestly, the idea to check a google XML sitemap sounds weird to me. Check the website, not its XML map. Tilman
          Message 4 of 4 , Apr 14, 2009
          • 0 Attachment
            Thomas is right :-)

            Honestly, the idea to check a google XML sitemap sounds weird to me.
            Check the website, not its XML map.

            Tilman

            On Tue, 14 Apr 2009 11:09:32 +0200, Thomas Fischer wrote:

            >
            >Hello!
            >
            >> One way to solve this is ( without asking Tilman for more
            >> development):
            >>
            >>
            >>
            >> 1. Download the target XML sitemap
            >>
            >> 2. Open it with Excel or similar.
            >> 3. Select the valid URLs and copy them
            >> 4. Create a file with the list of URLs, with one URL in each line.
            >> 5. Go to XENU and go to File > Check URL list ( test)
            >> 6. Open the file with URLs .
            >> 7. ready. :-)
            >>
            >
            >My understanding is that this won't work as intended.
            >The "File > Check URL list ( test)" is a multiple "Check URL" feature, which
            >would spawn separate crawlings for each URL, which is not the goal here.
            >If you want to check a list of URLs, the only way seems to be to create
            >links
            ><A HREF="...">Anything</A>
            >and store them in a file. Then use the "Check URL..." with the "Local
            >File..." option. You must mark the "Check external links" checkbox, because
            >all the URLs you give are external to the local file, and you might end up
            >with some URLs checked that are more remote than you expected.
            >By the way, I would use a text editor to extract the URLs from the sitemap
            >and create the links; on Windows Notepad++ will do nicely, as will
            >TextWrangler on a Mac.
            >
            >All the best
            >Thomas
            >
            >
            >
            >------------------------------------
            >
            >Yahoo! Groups Links
            >
            >
            >
          Your message has been successfully submitted and would be delivered to recipients shortly.