Loading ...
Sorry, an error occurred while loading the content.
 

Re: [xenu-usergroup] Spider sitemap

Expand Messages
  • Ignacio Seron
    One way to solve this is ( without asking Tilman for more development): 1. Download the target XML sitemap 2. Open it with Excel or similar. 3. Select the
    Message 1 of 4 , Apr 14, 2009
      One way to solve this is ( without asking Tilman for more development):

      1. Download the target XML sitemap
      2. Open it with Excel or similar.
      3. Select the valid URLs and copy them
      4. Create a file with the list of URLs, with one URL in each line.
      5. Go to XENU and go to File > Check URL list ( test)
      6. Open the file with URLs .
      7. ready. :-)
      It is not so complicated to to it.
      regards
      Ignacio



      2009/4/14 Nick
      <noodles@...>



      Can Xenu take a Google sitemap as an input and spider all pages listed and check for broken links (both in the sitemap and another level deep)?

      If not, this would be a great feature. I run a few sites where if I let Xenu loose on the site it will take days to finish spidering. Giving it a list of URLs would be much simpler.

      Thanks
      Nick


    • Thomas Fischer
      Hello! ... My understanding is that this won t work as intended. The File Check URL list ( test) is a multiple Check URL feature, which would spawn
      Message 2 of 4 , Apr 14, 2009
        Hello!

        > One way to solve this is ( without asking Tilman for more
        > development):
        >
        >
        >
        > 1. Download the target XML sitemap
        >
        > 2. Open it with Excel or similar.
        > 3. Select the valid URLs and copy them
        > 4. Create a file with the list of URLs, with one URL in each line.
        > 5. Go to XENU and go to File > Check URL list ( test)
        > 6. Open the file with URLs .
        > 7. ready. :-)
        >

        My understanding is that this won't work as intended.
        The "File > Check URL list ( test)" is a multiple "Check URL" feature, which
        would spawn separate crawlings for each URL, which is not the goal here.
        If you want to check a list of URLs, the only way seems to be to create
        links
        <A HREF="...">Anything</A>
        and store them in a file. Then use the "Check URL..." with the "Local
        File..." option. You must mark the "Check external links" checkbox, because
        all the URLs you give are external to the local file, and you might end up
        with some URLs checked that are more remote than you expected.
        By the way, I would use a text editor to extract the URLs from the sitemap
        and create the links; on Windows Notepad++ will do nicely, as will
        TextWrangler on a Mac.

        All the best
        Thomas
      • Tilman Hausherr
        Thomas is right :-) Honestly, the idea to check a google XML sitemap sounds weird to me. Check the website, not its XML map. Tilman
        Message 3 of 4 , Apr 14, 2009
          Thomas is right :-)

          Honestly, the idea to check a google XML sitemap sounds weird to me.
          Check the website, not its XML map.

          Tilman

          On Tue, 14 Apr 2009 11:09:32 +0200, Thomas Fischer wrote:

          >
          >Hello!
          >
          >> One way to solve this is ( without asking Tilman for more
          >> development):
          >>
          >>
          >>
          >> 1. Download the target XML sitemap
          >>
          >> 2. Open it with Excel or similar.
          >> 3. Select the valid URLs and copy them
          >> 4. Create a file with the list of URLs, with one URL in each line.
          >> 5. Go to XENU and go to File > Check URL list ( test)
          >> 6. Open the file with URLs .
          >> 7. ready. :-)
          >>
          >
          >My understanding is that this won't work as intended.
          >The "File > Check URL list ( test)" is a multiple "Check URL" feature, which
          >would spawn separate crawlings for each URL, which is not the goal here.
          >If you want to check a list of URLs, the only way seems to be to create
          >links
          ><A HREF="...">Anything</A>
          >and store them in a file. Then use the "Check URL..." with the "Local
          >File..." option. You must mark the "Check external links" checkbox, because
          >all the URLs you give are external to the local file, and you might end up
          >with some URLs checked that are more remote than you expected.
          >By the way, I would use a text editor to extract the URLs from the sitemap
          >and create the links; on Windows Notepad++ will do nicely, as will
          >TextWrangler on a Mac.
          >
          >All the best
          >Thomas
          >
          >
          >
          >------------------------------------
          >
          >Yahoo! Groups Links
          >
          >
          >
        Your message has been successfully submitted and would be delivered to recipients shortly.