Loading ...
Sorry, an error occurred while loading the content.

23Re: [xenu-usergroup] External links

Expand Messages
  • andi
    Aug 1, 2004
    • 0 Attachment
      For your reference, I have a Comcast Broadband connection that averages around 2800kbps.  I have Xenu set at 30 threads.  I usually make up a special page with about 10,000 external links and run it very late at night.  The first pass usually yeilds about a third of the links "action cancelled" which is the ISP choking on the load.  I save the .XEN file and retry the broken links, Ctrl-R about an hour later and again I get a smaller portion of the remainder "action cancelled."  Usually it is good to redo this several times because some percentage of the links themselves will always be temporarily out of service at any given time.
       
      I set a 14 day interval between checks and so over the course of the two weeks I'm confident there are a minimal number of errors due to my own connection.  By that time there are usually just 25-30 dead links out of 10K which are easy enough to check manually.  I use the iRider http://www.irider.com/  browser which will open 25 links like a bot but presents itself as IE6 for the final final check thus eliminating sites blocking robot user-agents.
       
      Andi
       
       
      ----- Original Message -----
      Sent: Sunday, August 01, 2004 1:04 AM
      Subject: Re: [xenu-usergroup] External links

      Try set a lower amount of threads, and retry with CTRL-R.

      Tilman

      On Sun, 01 Aug 2004 09:01:46 +0100, barrie@... wrote:

      >Hi,
      >
      >I am sure that I am doing something incorrectly <grin>
      >
      >When I request a report from my actual web page many of my external
      >links are reported faulty, with a "no such host" comment.
      >
      >What I find odd is that each of these links works quite happily if I
      >double click the link in the error report itself, or in the web page
      >itself.
      >
      >I have a reasonably fast connection using DSL.
      >
      >Not all external links are reported faulty, perhaps about 50%
      >
      >The url of my links page is  http://www.clock-museum.co.uk/links.htm
      >
      >Any help will be most gratefully received. I find Xenu invaluable and
      >as I said above, I feel sure this problem is due to me being dim.
      >
      >
      >Barrie Smith (in sunny Kent UK)
      >
      >Visit my Virtual Clock Museum at http://www.clock-museum.co.uk/
      >
      >
      >
      >
      >
      >Yahoo! Groups Links
      >
      >
      >
      >


    • Show all 12 messages in this topic