Loading ...
Sorry, an error occurred while loading the content.

Re: [xenu-usergroup] if life will be a request programme....

Expand Messages
  • Tilman Hausherr
    No... because this would go against the wishes of those webmasters that don t want Xenu to access their sites. Tilman
    Message 1 of 8 , Sep 13, 2010
    • 0 Attachment
      No... because this would go against the wishes of those webmasters that
      don't want Xenu to access their sites.

      Tilman

      On Mon, 13 Sep 2010 15:19:35 -0000, Wolfgang wrote:

      >Hi Tilman,
      >... i know it isn´t but sometimes it looks like.
      >
      >Unlikely there is no regular opportunity to manupulate the http-Header of Xenu.
      >Therfore some pages return "forbidden request" instead of regular result during XENU touches them (e.g. http://de.wikipedia.org/wiki/RSS)
      >
      >Is ther some chance, to make XENU-Request-Headers temporary changeable by changeing options?
      >
      >greetings
      >Wolfgang
      >
      >
      >
      >------------------------------------
      >
      >Yahoo! Groups Links
      >
      >
      >
    • Ron Jones
      One would also ask why do people want to use Xenu on Wikipedia . Due to the way that Wikipedia is extensively cross linked, if Xenu did run, then you would
      Message 2 of 8 , Sep 14, 2010
      • 0 Attachment
        One would also ask why "do people want to use Xenu on Wikipedia". Due to the
        way that Wikipedia is extensively cross linked, if Xenu did run, then you
        would get *every* page on every Wikipedia in one go, which must have a
        impact on the server. If you want to download every page then there is a
        special way of obtaining a database dump (compressed) for each Wikipedia.

        Ron Jones

        ----- Original Message -----
        From: "Tilman Hausherr" <tilman@...>
        To: <xenu-usergroup@yahoogroups.com>
        Sent: Monday, September 13, 2010 5:05 PM
        Subject: Re: [xenu-usergroup] if life will be a request programme....


        > No... because this would go against the wishes of those webmasters that
        > don't want Xenu to access their sites.
        >
        > Tilman
        >
        > On Mon, 13 Sep 2010 15:19:35 -0000, Wolfgang wrote:
        >
        >>Hi Tilman,
        >>... i know it isn�t but sometimes it looks like.
        >>
        >>Unlikely there is no regular opportunity to manupulate the http-Header of
        >>Xenu.
        >>Therfore some pages return "forbidden request" instead of regular result
        >>during XENU touches them (e.g. http://de.wikipedia.org/wiki/RSS)
        >>
        >>Is ther some chance, to make XENU-Request-Headers temporary changeable by
        >>changeing options?
        >>
        >>greetings
        >>Wolfgang
        >>
        >>
        >>
        >>------------------------------------
        >>
        >>Yahoo! Groups Links
        >>
        >>
        >>
        >
        >
        > ------------------------------------
        >
        > Yahoo! Groups Links
        >
        >
        >
        >
      • Vince Thacker
        Well, some pages have the odd reference to Wikipedia - inevitable at times. Ditto things like Google |News, which usually also returns a 403. Personally I let
        Message 3 of 8 , Sep 14, 2010
        • 0 Attachment
          Well, some pages have the odd reference to Wikipedia - inevitable at times.
          Ditto things like Google |News, which usually also returns a 403. Personally
          I let it all happen but skip over all those 403 errors.

          Vince.

          ----- Original Message -----
          From: "Ron Jones" <ron@...>
          To: <xenu-usergroup@yahoogroups.com>
          Sent: Wednesday, September 15, 2010 1:57 AM
          Subject: Re: [xenu-usergroup] if life will be a request programme....


          >
          > One would also ask why "do people want to use Xenu on Wikipedia". Due to
          > the way that Wikipedia is extensively cross linked, if Xenu did run, then
          > you would get *every* page on every Wikipedia in one go, which must have a
          > impact on the server. If you want to download every page then there is a
          > special way of obtaining a database dump (compressed) for each Wikipedia.
          >
          > Ron Jones
          >
          > ----- Original Message -----
          > From: "Tilman Hausherr" <tilman@...>
          > To: <xenu-usergroup@yahoogroups.com>
          > Sent: Monday, September 13, 2010 5:05 PM
          > Subject: Re: [xenu-usergroup] if life will be a request programme....
          >
          >
          >> No... because this would go against the wishes of those webmasters that
          >> don't want Xenu to access their sites.
          >>
          >> Tilman
          >>
          >> On Mon, 13 Sep 2010 15:19:35 -0000, Wolfgang wrote:
          >>
          >>>Hi Tilman,
          >>>... i know it isn´t but sometimes it looks like.
          >>>
          >>>Unlikely there is no regular opportunity to manupulate the http-Header of
          >>>Xenu.
          >>>Therfore some pages return "forbidden request" instead of regular result
          >>>during XENU touches them (e.g. http://de.wikipedia.org/wiki/RSS)
          >>>
          >>>Is ther some chance, to make XENU-Request-Headers temporary changeable by
          >>>changeing options?
          >>>
          >>>greetings
          >>>Wolfgang
          >>>
          >>>
          >>>
          >>>------------------------------------
          >>>
          >>>Yahoo! Groups Links
          >>>
          >>>
          >>>
          >>
          >>
          >> ------------------------------------
          >>
          >> Yahoo! Groups Links
          >>
          >>
          >>
          >>
          >
          >
          >
          >
          > ------------------------------------
          >
          > Yahoo! Groups Links
          >
          >
          >
          >
        • Wolfgang
          Ok, wikipedia was my worst example (but the best known). We do handle several Link-List to forein friendly motorbike-websites (for example Clubs). We don´t
          Message 4 of 8 , Sep 14, 2010
          • 0 Attachment
            Ok, wikipedia was my worst example (but the best known).
            We do handle several Link-List to forein friendly motorbike-websites (for example Clubs).
            We don´t link www.mydomain.com eachtime, sometimes it si neccessary to link direct to a pag like www.mydomain/home.htm becaus of misconfiguration of der Webservers.
            If the do change their page-structure f.E. home.htm isn´t avalable index.html instead it´s easy for us to recognize and to change on return of 404. But returning "forbidden request" i have to check each page by opening in browser.

            best regard
            wolfgang

            --- In xenu-usergroup@yahoogroups.com, "Vince Thacker" <vince@...> wrote:
            >
            > Well, some pages have the odd reference to Wikipedia - inevitable at times.
            > Ditto things like Google |News, which usually also returns a 403. Personally
            > I let it all happen but skip over all those 403 errors.
            >
            > Vince.
            >
            > ----- Original Message -----
            > From: "Ron Jones" <ron@...>
            > To: <xenu-usergroup@yahoogroups.com>
            > Sent: Wednesday, September 15, 2010 1:57 AM
            > Subject: Re: [xenu-usergroup] if life will be a request programme....
            >
            >
            > >
            > > One would also ask why "do people want to use Xenu on Wikipedia". Due to
            > > the way that Wikipedia is extensively cross linked, if Xenu did run, then
            > > you would get *every* page on every Wikipedia in one go, which must have a
            > > impact on the server. If you want to download every page then there is a
            > > special way of obtaining a database dump (compressed) for each Wikipedia.
            > >
            > > Ron Jones
            > >
            > > ----- Original Message -----
            > > From: "Tilman Hausherr" <tilman@...>
            > > To: <xenu-usergroup@yahoogroups.com>
            > > Sent: Monday, September 13, 2010 5:05 PM
            > > Subject: Re: [xenu-usergroup] if life will be a request programme....
            > >
            > >
            > >> No... because this would go against the wishes of those webmasters that
            > >> don't want Xenu to access their sites.
            > >>
            > >> Tilman
            > >>
            > >> On Mon, 13 Sep 2010 15:19:35 -0000, Wolfgang wrote:
            > >>
            > >>>Hi Tilman,
            > >>>... i know it isn´t but sometimes it looks like.
            > >>>
            > >>>Unlikely there is no regular opportunity to manupulate the http-Header of
            > >>>Xenu.
            > >>>Therfore some pages return "forbidden request" instead of regular result
            > >>>during XENU touches them (e.g. http://de.wikipedia.org/wiki/RSS)
            > >>>
            > >>>Is ther some chance, to make XENU-Request-Headers temporary changeable by
            > >>>changeing options?
            > >>>
            > >>>greetings
            > >>>Wolfgang
            > >>>
            > >>>
            > >>>
            > >>>------------------------------------
            > >>>
            > >>>Yahoo! Groups Links
            > >>>
            > >>>
            > >>>
            > >>
            > >>
            > >> ------------------------------------
            > >>
            > >> Yahoo! Groups Links
            > >>
            > >>
            > >>
            > >>
            > >
            > >
            > >
            > >
            > > ------------------------------------
            > >
            > > Yahoo! Groups Links
            > >
            > >
            > >
            > >
            >
          • Tilman Hausherr
            You d need to talk to these websites... sometimes, its these errors are not even on purpose, but the result of poor php programming. I ve often have had
            Message 5 of 8 , Sep 15, 2010
            • 0 Attachment
              You'd need to talk to these websites... sometimes, its these errors are
              not even on purpose, but the result of poor php programming. I've often
              have had support requests from users who have php-based sites and have
              no idea what they're actually doing.

              Tilman

              On Wed, 15 Sep 2010 06:42:42 -0000, Wolfgang wrote:

              >Ok, wikipedia was my worst example (but the best known).
              >We do handle several Link-List to forein friendly motorbike-websites (for example Clubs).
              >We don´t link www.mydomain.com eachtime, sometimes it si neccessary to link direct to a pag like www.mydomain/home.htm becaus of misconfiguration of der Webservers.
              >If the do change their page-structure f.E. home.htm isn´t avalable index.html instead it´s easy for us to recognize and to change on return of 404. But returning "forbidden request" i have to check each page by opening in browser.
              >
              >best regard
              >wolfgang
              >
              >--- In xenu-usergroup@yahoogroups.com, "Vince Thacker" <vince@...> wrote:
              >>
              >> Well, some pages have the odd reference to Wikipedia - inevitable at times.
              >> Ditto things like Google |News, which usually also returns a 403. Personally
              >> I let it all happen but skip over all those 403 errors.
              >>
              >> Vince.
              >>
              >> ----- Original Message -----
              >> From: "Ron Jones" <ron@...>
              >> To: <xenu-usergroup@yahoogroups.com>
              >> Sent: Wednesday, September 15, 2010 1:57 AM
              >> Subject: Re: [xenu-usergroup] if life will be a request programme....
              >>
              >>
              >> >
              >> > One would also ask why "do people want to use Xenu on Wikipedia". Due to
              >> > the way that Wikipedia is extensively cross linked, if Xenu did run, then
              >> > you would get *every* page on every Wikipedia in one go, which must have a
              >> > impact on the server. If you want to download every page then there is a
              >> > special way of obtaining a database dump (compressed) for each Wikipedia.
              >> >
              >> > Ron Jones
              >> >
              >> > ----- Original Message -----
              >> > From: "Tilman Hausherr" <tilman@...>
              >> > To: <xenu-usergroup@yahoogroups.com>
              >> > Sent: Monday, September 13, 2010 5:05 PM
              >> > Subject: Re: [xenu-usergroup] if life will be a request programme....
              >> >
              >> >
              >> >> No... because this would go against the wishes of those webmasters that
              >> >> don't want Xenu to access their sites.
              >> >>
              >> >> Tilman
              >> >>
              >> >> On Mon, 13 Sep 2010 15:19:35 -0000, Wolfgang wrote:
              >> >>
              >> >>>Hi Tilman,
              >> >>>... i know it isn´t but sometimes it looks like.
              >> >>>
              >> >>>Unlikely there is no regular opportunity to manupulate the http-Header of
              >> >>>Xenu.
              >> >>>Therfore some pages return "forbidden request" instead of regular result
              >> >>>during XENU touches them (e.g. http://de.wikipedia.org/wiki/RSS)
              >> >>>
              >> >>>Is ther some chance, to make XENU-Request-Headers temporary changeable by
              >> >>>changeing options?
              >> >>>
              >> >>>greetings
              >> >>>Wolfgang
              >> >>>
              >> >>>
              >> >>>
              >> >>>------------------------------------
              >> >>>
              >> >>>Yahoo! Groups Links
              >> >>>
              >> >>>
              >> >>>
              >> >>
              >> >>
              >> >> ------------------------------------
              >> >>
              >> >> Yahoo! Groups Links
              >> >>
              >> >>
              >> >>
              >> >>
              >> >
              >> >
              >> >
              >> >
              >> > ------------------------------------
              >> >
              >> > Yahoo! Groups Links
              >> >
              >> >
              >> >
              >> >
              >>
              >
              >
              >
              >
              >------------------------------------
              >
              >Yahoo! Groups Links
              >
              >
              >
            • Wolfgang
              Hi Tilman, another little wish. I use Xenu successfully to check several sites. Unlikely XENU does only recognize the last 10 configuratons. To hold more
              Message 6 of 8 , Sep 20, 2010
              • 0 Attachment
                Hi Tilman,
                another little wish.
                I use Xenu successfully to check several sites.
                Unlikely XENU does only recognize the last 10 configuratons.
                To hold more configurations it is neccessary to have another copy of XENU.
                Is it possible to drill out XENU to recognize "n" configurations or to load and save different xenu.inis using one application?
                This will be very helpfull for me.

                best regards
                Wolfgang

                --- In xenu-usergroup@yahoogroups.com, Tilman Hausherr <tilman@...> wrote:
                >
                > You'd need to talk to these websites... sometimes, its these errors are
                > not even on purpose, but the result of poor php programming. I've often
                > have had support requests from users who have php-based sites and have
                > no idea what they're actually doing.
                >
                > Tilman
                >
                > On Wed, 15 Sep 2010 06:42:42 -0000, Wolfgang wrote:
                >
                > >Ok, wikipedia was my worst example (but the best known).
                > >We do handle several Link-List to forein friendly motorbike-websites (for example Clubs).
                > >We don´t link www.mydomain.com eachtime, sometimes it si neccessary to link direct to a pag like www.mydomain/home.htm becaus of misconfiguration of der Webservers.
                > >If the do change their page-structure f.E. home.htm isn´t avalable index.html instead it´s easy for us to recognize and to change on return of 404. But returning "forbidden request" i have to check each page by opening in browser.
                > >
                > >best regard
                > >wolfgang
                > >
                > >--- In xenu-usergroup@yahoogroups.com, "Vince Thacker" <vince@> wrote:
                > >>
                > >> Well, some pages have the odd reference to Wikipedia - inevitable at times.
                > >> Ditto things like Google |News, which usually also returns a 403. Personally
                > >> I let it all happen but skip over all those 403 errors.
                > >>
                > >> Vince.
                > >>
                > >> ----- Original Message -----
                > >> From: "Ron Jones" <ron@>
                > >> To: <xenu-usergroup@yahoogroups.com>
                > >> Sent: Wednesday, September 15, 2010 1:57 AM
                > >> Subject: Re: [xenu-usergroup] if life will be a request programme....
                > >>
                > >>
                > >> >
                > >> > One would also ask why "do people want to use Xenu on Wikipedia". Due to
                > >> > the way that Wikipedia is extensively cross linked, if Xenu did run, then
                > >> > you would get *every* page on every Wikipedia in one go, which must have a
                > >> > impact on the server. If you want to download every page then there is a
                > >> > special way of obtaining a database dump (compressed) for each Wikipedia.
                > >> >
                > >> > Ron Jones
                > >> >
                > >> > ----- Original Message -----
                > >> > From: "Tilman Hausherr" <tilman@>
                > >> > To: <xenu-usergroup@yahoogroups.com>
                > >> > Sent: Monday, September 13, 2010 5:05 PM
                > >> > Subject: Re: [xenu-usergroup] if life will be a request programme....
                > >> >
                > >> >
                > >> >> No... because this would go against the wishes of those webmasters that
                > >> >> don't want Xenu to access their sites.
                > >> >>
                > >> >> Tilman
                > >> >>
                > >> >> On Mon, 13 Sep 2010 15:19:35 -0000, Wolfgang wrote:
                > >> >>
                > >> >>>Hi Tilman,
                > >> >>>... i know it isn´t but sometimes it looks like.
                > >> >>>
                > >> >>>Unlikely there is no regular opportunity to manupulate the http-Header of
                > >> >>>Xenu.
                > >> >>>Therfore some pages return "forbidden request" instead of regular result
                > >> >>>during XENU touches them (e.g. http://de.wikipedia.org/wiki/RSS)
                > >> >>>
                > >> >>>Is ther some chance, to make XENU-Request-Headers temporary changeable by
                > >> >>>changeing options?
                > >> >>>
                > >> >>>greetings
                > >> >>>Wolfgang
                > >> >>>
                > >> >>>
                > >> >>>
                > >> >>>------------------------------------
                > >> >>>
                > >> >>>Yahoo! Groups Links
                > >> >>>
                > >> >>>
                > >> >>>
                > >> >>
                > >> >>
                > >> >> ------------------------------------
                > >> >>
                > >> >> Yahoo! Groups Links
                > >> >>
                > >> >>
                > >> >>
                > >> >>
                > >> >
                > >> >
                > >> >
                > >> >
                > >> > ------------------------------------
                > >> >
                > >> > Yahoo! Groups Links
                > >> >
                > >> >
                > >> >
                > >> >
                > >>
                > >
                > >
                > >
                > >
                > >------------------------------------
                > >
                > >Yahoo! Groups Links
                > >
                > >
                > >
                >
              • Tilman Hausherr
                Get the latest version, which shows more than 10 URLs in the dialog. However even the previous versions did remember more than 10 configurations. (Through the
                Message 7 of 8 , Sep 20, 2010
                • 0 Attachment
                  Get the latest version, which shows more than 10 URLs in the dialog.
                  However even the previous versions did remember more than 10
                  configurations. (Through the INI file)

                  Tilman

                  On Mon, 20 Sep 2010 08:57:46 -0000, Wolfgang <w.peters@...> wrote:

                  > Hi Tilman,
                  > another little wish.
                  > I use Xenu successfully to check several sites.
                  > Unlikely XENU does only recognize the last 10 configuratons.
                  > To hold more configurations it is neccessary to have another copy of
                  > XENU. Is it possible to drill out XENU to recognize "n" configurations
                  > or to load and save different xenu.inis using one application? This will
                  > be very helpfull for me.
                  >
                  > best regards
                  > Wolfgang
                  >
                  > --- In xenu-usergroup@yahoogroups.com, Tilman Hausherr <tilman@...>
                  > wrote:
                  > >
                  > > You'd need to talk to these websites... sometimes, its these errors
                  > > are not even on purpose, but the result of poor php programming. I've
                  > > often have had support requests from users who have php-based sites
                  > > and have no idea what they're actually doing.
                  > >
                  > > Tilman
                  > >
                  > > On Wed, 15 Sep 2010 06:42:42 -0000, Wolfgang wrote:
                  > >
                  > > >Ok, wikipedia was my worst example (but the best known).
                  > > >We do handle several Link-List to forein friendly motorbike-websites
                  > > >(for example Clubs). We don't link www.mydomain.com eachtime,
                  > > >sometimes it si neccessary to link direct to a pag like
                  > > >www.mydomain/home.htm becaus of misconfiguration of der Webservers.
                  > > >If the do change their page-structure f.E. home.htm isn't avalable
                  > > >index.html instead it's easy for us to recognize and to change on
                  > > >return of 404. But returning "forbidden request" i have to check each
                  > > >page by opening in browser.
                  > > >
                  > > >best regard
                  > > >wolfgang
                  > > >
                  > > >--- In xenu-usergroup@yahoogroups.com, "Vince Thacker" <vince@>
                  > > >wrote:
                  > > >>
                  > > >> Well, some pages have the odd reference to Wikipedia - inevitable
                  > > >> at times. Ditto things like Google |News, which usually also
                  > > >> returns a 403. Personally I let it all happen but skip over all
                  > > >> those 403 errors.
                  > > >>
                  > > >> Vince.
                  > > >>
                  > > >> ----- Original Message -----
                  > > >> From: "Ron Jones" <ron@>
                  > > >> To: <xenu-usergroup@yahoogroups.com>
                  > > >> Sent: Wednesday, September 15, 2010 1:57 AM
                  > > >> Subject: Re: [xenu-usergroup] if life will be a request
                  > > >> programme....
                  > > >>
                  > > >> >
                  > > >> > One would also ask why "do people want to use Xenu on Wikipedia".
                  > > >> > Due to the way that Wikipedia is extensively cross linked, if
                  > > >> > Xenu did run, then you would get *every* page on every Wikipedia
                  > > >> > in one go, which must have a impact on the server. If you want
                  > > >> > to download every page then there is a special way of obtaining
                  > > >> > a database dump (compressed) for each Wikipedia.
                  > > >> >
                  > > >> > Ron Jones
                  > > >> >
                  > > >> > ----- Original Message -----
                  > > >> > From: "Tilman Hausherr" <tilman@>
                  > > >> > To: <xenu-usergroup@yahoogroups.com>
                  > > >> > Sent: Monday, September 13, 2010 5:05 PM
                  > > >> > Subject: Re: [xenu-usergroup] if life will be a request
                  > > >> > programme....
                  > > >> >
                  > > >> >
                  > > >> >> No... because this would go against the wishes of those
                  > > >> >> webmasters that don't want Xenu to access their sites.
                  > > >> >>
                  > > >> >> Tilman
                  > > >> >>
                  > > >> >> On Mon, 13 Sep 2010 15:19:35 -0000, Wolfgang wrote:
                  > > >> >>
                  > > >> >>>Hi Tilman,
                  > > >> >>>... i know it isn't but sometimes it looks like.
                  > > >> >>>
                  > > >> >>>Unlikely there is no regular opportunity to manupulate the
                  > > >> >>>http-Header of Xenu.
                  > > >> >>>Therfore some pages return "forbidden request" instead of
                  > > >> >>>regular result during XENU touches them (e.g.
                  > > >> >>>http://de.wikipedia.org/wiki/RSS)
                  > > >> >>>
                  > > >> >>>Is ther some chance, to make XENU-Request-Headers temporary
                  > > >> >>>changeable by changeing options?
                  > > >> >>>
                  > > >> >>>greetings
                  > > >> >>>Wolfgang
                  > > >> >>>
                  > > >> >>>------------------------------------
                  > > >> >>>
                  > > >> >>>Yahoo! Groups Links
                  > > >> >>>
                  > > >> >>
                  > > >> >>
                  > > >> >> ------------------------------------
                  > > >> >>
                  > > >> >> Yahoo! Groups Links
                  > > >> >>
                  > > >> >>
                  > > >> >
                  > > >> >
                  > > >> > ------------------------------------
                  > > >> >
                  > > >> > Yahoo! Groups Links
                  > > >> >
                  > > >> >
                  > > >>
                  > > >
                  > > >
                  > > >------------------------------------
                  > > >
                  > > >Yahoo! Groups Links
                  > > >
                  > >
                  >
                  >
                  > ------------------------------------
                  >
                  > Yahoo! Groups Links
                  >
                  >
                  >
                Your message has been successfully submitted and would be delivered to recipients shortly.