Re: [xenu-usergroup] False negatives and other knotty bits
- Thanks a lot -- I think I've got it now!Vince.----- Original Message -----From: Tilman HausherrSent: Wednesday, February 14, 2007 5:41 PMSubject: Re: [xenu-usergroup] False negatives and other knotty bits
On Wed, 14 Feb 2007 16:06:01 -0000, Vince wrote:
>Some questions I would like to raise about the results I get from
>Firstly, more and more sites seem to be giving out a 404 not found
>error when the page actually is there, and looks fine when visited.
See FAQ - some websites "hate" the Xenu user agent
>Same story with the 'no such host' message, which sometimes turns
>out not to be true. Why does this happen? I take it that we're dealing
This could be a network problem. Try CTRL-R. Also, be sure to have the
>with some behaviour of web hosts here, as much as the Xenu
>My second question is about links that appear at the end of the 'By
>Page' list in the Xenu report. Some of these are not links on my site
>at all. This seems to arise from memberships at Webring.com - Xenu
>seems to follow up links that are outside my site but may be in the
These are redirections. Solution: exclude the webring from the spidering
in the initial dialogbox.
>a little enlightenment on these points would be very good. Is there
>any setting I can change to address either of these questions?
>Apart from these queries, I have to say Xenu is indispensable on my
>system, and must have saved me hundreds of hours of grief when
>Yahoo! Groups Links