Loading ...
Sorry, an error occurred while loading the content.
 

Re: [xenu-usergroup] Wikipedia links not working for Xenu?

Expand Messages
  • Ken Lorimer
    This is an excellent summary of Professor Whitehead. His work with Bertrand Russel on the theory of classes was a breakthrough at that time. Both men are of
    Message 1 of 6 , Jul 14, 2005
      This is an excellent summary of Professor Whitehead. His work with Bertrand Russel on the theory of classes was a breakthrough at that time. Both men are of high intellect. I studied their writings at university and I am pleased that you cited them.
      Thank you.

      Ken Lorimer.

      Tilman Hausherr <tilman@...> wrote:
      On Thu, 07 Jul 2005 13:04:39 -0000, frank visser wrote:

      >do you know why Wikipedia links of the kind:
      >http://en.wikipedia.org/wiki/Alfred_North_Whitehead
      >
      >are not working for xenu?

      http://home.snafu.de/tilman/xenulink.html

      See Nr. 20





      ---------------------------------
      YAHOO! GROUPS LINKS


      Visit your group "xenu-usergroup" on the web.

      To unsubscribe from this group, send an email to:
      xenu-usergroup-unsubscribe@yahoogroups.com

      Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.


      ---------------------------------




      ---------------------------------
      Start your day with Yahoo! - make it your home page

      [Non-text portions of this message have been removed]
    • adiscretemale
      Andi, I checked your page and noted that you say There is one major company URL (http://www.prudential.com/) that will recycle through dozens of redirects for
      Message 2 of 6 , Jul 25, 2005
        Andi, I checked your page and noted that you say "There is one major
        company URL (http://www.prudential.com/) that will recycle through
        dozens of redirects for Xenu yet behaves normally for a browser so I
        simply omit it as well." Out of curiousity, I tried starting with
        that URL with Xenu and it only went through 3 redirects before
        actually going into the site. The redirects appear to be the same as
        those a browser gets since if you enter http://www.prudential.com/ in
        browser, when the page comes up, the URL shows as
        http://www.prudential.com/index/.

        Investigating further, I determined that
        http://www.prudential.com/
        first redirects to
        http://www.prudential.com/default/
        that, in turn, redirects to
        http://www.prudential.com/index
        and that, again, redirects to
        http://www.prudential.com/index/
        and that returns a normal page.

        Tom
        --- In xenu-usergroup@yahoogroups.com, "andi" <andi@a...> wrote:
        > I've long had this same experience with Wikipedia using other bots
        as well as Xenu. (returns "Wikimedia error") I have always assumed
        that Wikipedia blocks anything that even slightly smells like a bot
        to keep their content from being spidered. Irresponsible bots are
        becoming so numerous that I'd like to do this on my site as well but
        such blocking is more difficult with static pages. I do find that my
        list of sites seeming to block Xenu continually grows (partial list
        here http://www.andilinks.com/linkckg.htm#55000).
        >
        > Andi
      • andi
        Thanks for this Tom. When I wrote that (It has been over a year) and put Prudential on the exeception list it was returning dozens of redirects to the Xenu
        Message 3 of 6 , Jul 26, 2005
          Thanks for this Tom. When I wrote that (It has been over a year) and put Prudential on the exeception list it was returning dozens of redirects to the Xenu result page. They must have fixed the bug in their rewrite rule (or whatever, perhaps there's a change in the Xenu version) in the meantime. I'll revise the text on my Link checking page. Thanks again.

          Andi
          ----- Original Message -----
          From: adiscretemale
          To: xenu-usergroup@yahoogroups.com
          Sent: Monday, July 25, 2005 3:57 AM
          Subject: [xenu-usergroup] Re: Wikipedia links not working for Xenu?


          Andi, I checked your page and noted that you say "There is one major
          company URL (http://www.prudential.com/) that will recycle through
          dozens of redirects for Xenu yet behaves normally for a browser so I
          simply omit it as well." Out of curiousity, I tried starting with
          that URL with Xenu and it only went through 3 redirects before
          actually going into the site. The redirects appear to be the same as
          those a browser gets since if you enter http://www.prudential.com/ in
          browser, when the page comes up, the URL shows as
          http://www.prudential.com/index/.

          Investigating further, I determined that
          http://www.prudential.com/
          first redirects to
          http://www.prudential.com/default/
          that, in turn, redirects to
          http://www.prudential.com/index
          and that, again, redirects to
          http://www.prudential.com/index/
          and that returns a normal page.

          Tom
          --- In xenu-usergroup@yahoogroups.com, "andi" <andi@a...> wrote:
          > I've long had this same experience with Wikipedia using other bots
          as well as Xenu. (returns "Wikimedia error") I have always assumed
          that Wikipedia blocks anything that even slightly smells like a bot
          to keep their content from being spidered. Irresponsible bots are
          becoming so numerous that I'd like to do this on my site as well but
          such blocking is more difficult with static pages. I do find that my
          list of sites seeming to block Xenu continually grows (partial list
          here http://www.andilinks.com/linkckg.htm#55000).
          >
          > Andi





          ------------------------------------------------------------------------------
          YAHOO! GROUPS LINKS

          a.. Visit your group "xenu-usergroup" on the web.

          b.. To unsubscribe from this group, send an email to:
          xenu-usergroup-unsubscribe@yahoogroups.com

          c.. Your use of Yahoo! Groups is subject to the Yahoo! Terms of Service.


          ------------------------------------------------------------------------------



          [Non-text portions of this message have been removed]
        Your message has been successfully submitted and would be delivered to recipients shortly.