Loading ...
Sorry, an error occurred while loading the content.

Re: [xenu-usergroup] Not Checking All Links

Expand Messages
  • Michael Hunsaker
    Thanks for taking a look at this for me! I appreciate it! Mike
    Message 1 of 10 , Oct 5, 2005
    • 0 Attachment
      Thanks for taking a look at this for me! I appreciate
      it!

      Mike

      --- Tilman Hausherr <tilman@...> wrote:

      > On Wed, 5 Oct 2005 09:30:29 -0700 (PDT), Michael
      > Hunsaker wrote:
      >
      > >#4 - http://www.usi.edu/alumni/connect/wolfe.asp
      >
      > I see it returns ok. Although the title is "404
      > error". This is a
      > configuation problem.
      >
      > Here is what I get when telneting to your server on
      > port 80 with a bogus
      > URL:
      >
      >
      > HTTP/1.1 200 OK
      > Date: Wed, 05 Oct 2005 21:24:38 GMT
      > Server: Microsoft-IIS/6.0
      > X-Powered-By: ASP.NET
      > Content-Length: 10144
      > Content-Type: text/html
      > Set-Cookie:
      > ASPSESSIONIDQCCTTTAT=KOGCGHACGDDIKDCIGNPNODMP;
      > path=/
      > Cache-control: private
      >
      >
      > <html>
      > <head>
      > <title>404 Error</title>
      > ....
      >
      >
      >
      >
      >
    • Tilman Hausherr
      ... See also Nr. 32 on http://home.snafu.de/tilman/xenulink.html, it explains what one IIS admin did to solve what seems to be the same problem. Tilman
      Message 2 of 10 , Oct 6, 2005
      • 0 Attachment
        On Wed, 5 Oct 2005 15:31:40 -0700 (PDT), Michael Hunsaker wrote:

        >Thanks for taking a look at this for me! I appreciate
        >it!

        See also Nr. 32 on http://home.snafu.de/tilman/xenulink.html, it
        explains what one IIS admin did to solve what seems to be the same
        problem.

        Tilman

        >
        >Mike
        >
        >--- Tilman Hausherr <tilman@...> wrote:
        >
        >> On Wed, 5 Oct 2005 09:30:29 -0700 (PDT), Michael
        >> Hunsaker wrote:
        >>
        >> >#4 - http://www.usi.edu/alumni/connect/wolfe.asp
        >>
        >> I see it returns ok. Although the title is "404
        >> error". This is a
        >> configuation problem.
        >>
        >> Here is what I get when telneting to your server on
        >> port 80 with a bogus
        >> URL:
        >>
        >>
        >> HTTP/1.1 200 OK
        >> Date: Wed, 05 Oct 2005 21:24:38 GMT
        >> Server: Microsoft-IIS/6.0
        >> X-Powered-By: ASP.NET
        >> Content-Length: 10144
        >> Content-Type: text/html
        >> Set-Cookie:
        >> ASPSESSIONIDQCCTTTAT=KOGCGHACGDDIKDCIGNPNODMP;
        >> path=/
        >> Cache-control: private
        >>
        >>
        >> <html>
        >> <head>
        >> <title>404 Error</title>
        >> ....
        >>
        >>
        >>
        >>
        >>
        >
        >
        >
        >
        >
        >
        >
        >Yahoo! Groups Links
        >
        >
        >
        >
        >
        >
        >
        >
      • Michael Hunsaker
        Frank, Thanks for your response! Do you happen to know why these pages are sending out an HTTP 200 status code instead of a HTTP 404? Is there something I
        Message 3 of 10 , Oct 6, 2005
        • 0 Attachment
          Frank,

          Thanks for your response! Do you happen to know why
          these pages are sending out an HTTP 200 status code
          instead of a HTTP 404? Is there something I can do
          within IIS to fix this?

          Thanks again,

          Mike

          --- Frank Visser <f.visser3@...> wrote:

          > Hi mike,
          >
          >
          >
          > That's because your server gives back a 200 (=ok)
          > message, and Xenu picks
          > this up:
          >
          >
          >
          > GET /alumni/connect/wolfe.asp HTTP/1.1
          >
          > Host: www.usi.edu
          >
          > User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1;
          > en-US; rv:1.7.5)
          > Gecko/20041107 Firefox/1.0
          >
          > Accept:
          >
          text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=
          > 0.8,image/png,*/*;q=0.5
          >
          > Accept-Language: en-us,en;q=0.5
          >
          > Accept-Encoding: gzip,deflate
          >
          > Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
          >
          > Keep-Alive: 300
          >
          > Connection: keep-alive
          >
          > Cookie:
          > ASPSESSIONIDQCCTTTAT=FFECGHACMPNAHNMNLHLHGHKO
          >
          > Cache-Control: max-age=0
          >
          >
          >
          > HTTP/1.x 200 OK
          >
          > Date: Wed, 05 Oct 2005 21:03:59 GMT
          >
          > Server: Microsoft-IIS/6.0
          >
          > X-Powered-By: ASP.NET
          >
          > Content-Length: 10174
          >
          > Content-Type: text/html
          >
          > Cache-Control: private
          >
          >
          >
          > It should return a 404 message, so it's a server
          > configuration issue.
          >
          >
          >
          > frank
          >
          >
          >
          > ========================================
          >
          > Frank Visser, Waterpoortweg 279, 1051 pv, Amsterdam
          >
          > Author of: Ken Wilber: Thought as Passion SUNY 2003
          >
          > Read all about Ken Wilber : http://
          > <http://www.integralworld.net>
          > www.integralworld.net
          >
          > ========================================
          >
          > _____
          >
          > From: xenu-usergroup@yahoogroups.com
          > [mailto:xenu-usergroup@yahoogroups.com]
          > On Behalf Of Michael Hunsaker
          > Sent: woensdag 5 oktober 2005 18:30
          > To: xenu-usergroup@yahoogroups.com
          > Subject: Re: [xenu-usergroup] Not Checking All Links
          >
          >
          >
          > Thanks for your response. Below is a path
          > navigation
          > to a file that does not exist but Xenu is not
          > picking
          > it up:
          >
          > #1 - http://www.usi.edu
          > #2 - http://www.usi.edu/alumni/index.asp
          > #3 - http://www.usi.edu/alumni/connect/index.asp
          > #4 - http://www.usi.edu/alumni/connect/wolfe.asp
          >
          > #4 is the broken link.
          >
          > Here is another example:
          >
          > #1 - http://www.usi.edu
          > #2 - http://www.usi.edu/science/index.asp
          > #3 - http://www.usi.edu/science/about.asp
          > #4 - http://www.usi.edu/science/Dean/page2.asp
          > #5 -
          >
          http://www.usi.edu/science/Dean/dean/CollegeSeminarSeries.asp
          >
          > #5 is the broken link.
          >
          > Those are just a couple of examples that I could
          > quickly come up with.
          >
          > Thanks again,
          >
          > Mike
          >
          >
          > --- Tilman Hausherr <tilman@...> wrote:
          >
          > > On Wed, 5 Oct 2005 06:04:40 -0700 (PDT), Michael
          > > Hunsaker wrote:
          > >
          > > >Sure, some of the broken links are not "real"
          > > broken
          > > >links but a good chunk of them are truly broken
          > > links.
          > > > Why isn't Xenu picking these up?
          > >
          > > Tell me a link that isn't spidered, but it linked
          > > from a page that is.
          > >
          > >
          >
          >
          >
          >
          >
          >
          > _____
          >
          > YAHOO! GROUPS LINKS
          >
          >
          >
          > * Visit your group "xenu-usergroup
          > <http://groups.yahoo.com/group/xenu-usergroup> " on
          > the web.
          >
          > * To unsubscribe from this group, send an email to:
          > xenu-usergroup-unsubscribe@yahoogroups.com
          >
          <mailto:xenu-usergroup-unsubscribe@yahoogroups.com?subject=Unsubscribe>
          >
          >
          > * Your use of Yahoo! Groups is subject to the
          > Yahoo!
          > <http://docs.yahoo.com/info/terms/> Terms of
          > Service.
          >
          >
          >
          > _____
          >
          >
          >
          > [Non-text portions of this message have been
          > removed]
          >
          >
        • frank visser
          mike, ... sent me an email each time a user hit the 404 error page this error page gives back a 200 instead of a 404, you should report this to your server
          Message 4 of 10 , Oct 7, 2005
          • 0 Attachment
            mike,

            as you said, the issue started when you added the error page:

            > I implemented a customized 404 error page that
            sent me an email each time a user hit the 404 error
            page

            this error page gives back a 200 instead of a 404, you should report
            this to your server administrator.

            can you see if this page perhaps redirects to a 404?
            http://www.usi.edu/alumni/connect/wolfe.asp

            i don't know enough about servers, but it seems a simple fix to me.

            frank



            --- In xenu-usergroup@yahoogroups.com, Michael Hunsaker
            <to_the_hurricave@y...> wrote:
            >
            > Frank,
            >
            > Thanks for your response! Do you happen to know why
            > these pages are sending out an HTTP 200 status code
            > instead of a HTTP 404? Is there something I can do
            > within IIS to fix this?
            >
            > Thanks again,
            >
            > Mike
            >
            > --- Frank Visser <f.visser3@c...> wrote:
            >
            > > Hi mike,
            > >
            > >
            > >
            > > That's because your server gives back a 200 (=ok)
            > > message, and Xenu picks
            > > this up:
            > >
            > >
            > >
            > > GET /alumni/connect/wolfe.asp HTTP/1.1
            > >
            > > Host: www.usi.edu
            > >
            > > User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1;
            > > en-US; rv:1.7.5)
            > > Gecko/20041107 Firefox/1.0
            > >
            > > Accept:
            > >
            >
            text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/pl
            ain;q=
            > > 0.8,image/png,*/*;q=0.5
            > >
            > > Accept-Language: en-us,en;q=0.5
            > >
            > > Accept-Encoding: gzip,deflate
            > >
            > > Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
            > >
            > > Keep-Alive: 300
            > >
            > > Connection: keep-alive
            > >
            > > Cookie:
            > > ASPSESSIONIDQCCTTTAT=FFECGHACMPNAHNMNLHLHGHKO
            > >
            > > Cache-Control: max-age=0
            > >
            > >
            > >
            > > HTTP/1.x 200 OK
            > >
            > > Date: Wed, 05 Oct 2005 21:03:59 GMT
            > >
            > > Server: Microsoft-IIS/6.0
            > >
            > > X-Powered-By: ASP.NET
            > >
            > > Content-Length: 10174
            > >
            > > Content-Type: text/html
            > >
            > > Cache-Control: private
            > >
            > >
            > >
            > > It should return a 404 message, so it's a server
            > > configuration issue.
            > >
            > >
            > >
            > > frank
            > >
            > >
            > >
            > > ========================================
            > >
            > > Frank Visser, Waterpoortweg 279, 1051 pv, Amsterdam
            > >
            > > Author of: Ken Wilber: Thought as Passion SUNY 2003
            > >
            > > Read all about Ken Wilber : http://
            > > <http://www.integralworld.net>
            > > www.integralworld.net
            > >
            > > ========================================
            > >
            > > _____
            > >
            > > From: xenu-usergroup@yahoogroups.com
            > > [mailto:xenu-usergroup@yahoogroups.com]
            > > On Behalf Of Michael Hunsaker
            > > Sent: woensdag 5 oktober 2005 18:30
            > > To: xenu-usergroup@yahoogroups.com
            > > Subject: Re: [xenu-usergroup] Not Checking All Links
            > >
            > >
            > >
            > > Thanks for your response. Below is a path
            > > navigation
            > > to a file that does not exist but Xenu is not
            > > picking
            > > it up:
            > >
            > > #1 - http://www.usi.edu
            > > #2 - http://www.usi.edu/alumni/index.asp
            > > #3 - http://www.usi.edu/alumni/connect/index.asp
            > > #4 - http://www.usi.edu/alumni/connect/wolfe.asp
            > >
            > > #4 is the broken link.
            > >
            > > Here is another example:
            > >
            > > #1 - http://www.usi.edu
            > > #2 - http://www.usi.edu/science/index.asp
            > > #3 - http://www.usi.edu/science/about.asp
            > > #4 - http://www.usi.edu/science/Dean/page2.asp
            > > #5 -
            > >
            > http://www.usi.edu/science/Dean/dean/CollegeSeminarSeries.asp
            > >
            > > #5 is the broken link.
            > >
            > > Those are just a couple of examples that I could
            > > quickly come up with.
            > >
            > > Thanks again,
            > >
            > > Mike
            > >
            > >
            > > --- Tilman Hausherr <tilman@s...> wrote:
            > >
            > > > On Wed, 5 Oct 2005 06:04:40 -0700 (PDT), Michael
            > > > Hunsaker wrote:
            > > >
            > > > >Sure, some of the broken links are not "real"
            > > > broken
            > > > >links but a good chunk of them are truly broken
            > > > links.
            > > > > Why isn't Xenu picking these up?
            > > >
            > > > Tell me a link that isn't spidered, but it linked
            > > > from a page that is.
            > > >
            > > >
            > >
            > >
            > >
            > >
            > >
            > >
            > > _____
            > >
            > > YAHOO! GROUPS LINKS
            > >
            > >
            > >
            > > * Visit your group "xenu-usergroup
            > > <http://groups.yahoo.com/group/xenu-usergroup> " on
            > > the web.
            > >
            > > * To unsubscribe from this group, send an email to:
            > > xenu-usergroup-unsubscribe@yahoogroups.com
            > >
            > <mailto:xenu-usergroup-unsubscribe@yahoogroups.com?
            subject=Unsubscribe>
            > >
            > >
            > > * Your use of Yahoo! Groups is subject to the
            > > Yahoo!
            > > <http://docs.yahoo.com/info/terms/> Terms of
            > > Service.
            > >
            > >
            > >
            > > _____
            > >
            > >
            > >
            > > [Non-text portions of this message have been
            > > removed]
            > >
            > >
            >
          • Michael Hunsaker
            I figured it out. Thanks Frank! I appreciate your feedback and help. Mike ... text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/pl ...
            Message 5 of 10 , Oct 7, 2005
            • 0 Attachment
              I figured it out. Thanks Frank! I appreciate your
              feedback and help.

              Mike

              --- frank visser <f.visser3@...> wrote:

              > mike,
              >
              > as you said, the issue started when you added the
              > error page:
              >
              > > I implemented a customized 404 error page that
              > sent me an email each time a user hit the 404 error
              > page
              >
              > this error page gives back a 200 instead of a 404,
              > you should report
              > this to your server administrator.
              >
              > can you see if this page perhaps redirects to a 404?
              > http://www.usi.edu/alumni/connect/wolfe.asp
              >
              > i don't know enough about servers, but it seems a
              > simple fix to me.
              >
              > frank
              >
              >
              >
              > --- In xenu-usergroup@yahoogroups.com, Michael
              > Hunsaker
              > <to_the_hurricave@y...> wrote:
              > >
              > > Frank,
              > >
              > > Thanks for your response! Do you happen to know
              > why
              > > these pages are sending out an HTTP 200 status
              > code
              > > instead of a HTTP 404? Is there something I can
              > do
              > > within IIS to fix this?
              > >
              > > Thanks again,
              > >
              > > Mike
              > >
              > > --- Frank Visser <f.visser3@c...> wrote:
              > >
              > > > Hi mike,
              > > >
              > > >
              > > >
              > > > That's because your server gives back a 200
              > (=ok)
              > > > message, and Xenu picks
              > > > this up:
              > > >
              > > >
              > > >
              > > > GET /alumni/connect/wolfe.asp HTTP/1.1
              > > >
              > > > Host: www.usi.edu
              > > >
              > > > User-Agent: Mozilla/5.0 (Windows; U; Windows NT
              > 5.1;
              > > > en-US; rv:1.7.5)
              > > > Gecko/20041107 Firefox/1.0
              > > >
              > > > Accept:
              > > >
              > >
              >
              text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/pl
              > ain;q=
              > > > 0.8,image/png,*/*;q=0.5
              > > >
              > > > Accept-Language: en-us,en;q=0.5
              > > >
              > > > Accept-Encoding: gzip,deflate
              > > >
              > > > Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
              > > >
              > > > Keep-Alive: 300
              > > >
              > > > Connection: keep-alive
              > > >
              > > > Cookie:
              > > > ASPSESSIONIDQCCTTTAT=FFECGHACMPNAHNMNLHLHGHKO
              > > >
              > > > Cache-Control: max-age=0
              > > >
              > > >
              > > >
              > > > HTTP/1.x 200 OK
              > > >
              > > > Date: Wed, 05 Oct 2005 21:03:59 GMT
              > > >
              > > > Server: Microsoft-IIS/6.0
              > > >
              > > > X-Powered-By: ASP.NET
              > > >
              > > > Content-Length: 10174
              > > >
              > > > Content-Type: text/html
              > > >
              > > > Cache-Control: private
              > > >
              > > >
              > > >
              > > > It should return a 404 message, so it's a server
              > > > configuration issue.
              > > >
              > > >
              > > >
              > > > frank
              > > >
              > > >
              > > >
              > > > ========================================
              > > >
              > > > Frank Visser, Waterpoortweg 279, 1051 pv,
              > Amsterdam
              > > >
              > > > Author of: Ken Wilber: Thought as Passion SUNY
              > 2003
              > > >
              > > > Read all about Ken Wilber : http://
              > > > <http://www.integralworld.net>
              > > > www.integralworld.net
              > > >
              > > > ========================================
              > > >
              > > > _____
              > > >
              > > > From: xenu-usergroup@yahoogroups.com
              > > > [mailto:xenu-usergroup@yahoogroups.com]
              > > > On Behalf Of Michael Hunsaker
              > > > Sent: woensdag 5 oktober 2005 18:30
              > > > To: xenu-usergroup@yahoogroups.com
              > > > Subject: Re: [xenu-usergroup] Not Checking All
              > Links
              > > >
              > > >
              > > >
              > > > Thanks for your response. Below is a path
              > > > navigation
              > > > to a file that does not exist but Xenu is not
              > > > picking
              > > > it up:
              > > >
              > > > #1 - http://www.usi.edu
              > > > #2 - http://www.usi.edu/alumni/index.asp
              > > > #3 - http://www.usi.edu/alumni/connect/index.asp
              > > > #4 - http://www.usi.edu/alumni/connect/wolfe.asp
              > > >
              > > > #4 is the broken link.
              > > >
              > > > Here is another example:
              > > >
              > > > #1 - http://www.usi.edu
              > > > #2 - http://www.usi.edu/science/index.asp
              > > > #3 - http://www.usi.edu/science/about.asp
              > > > #4 - http://www.usi.edu/science/Dean/page2.asp
              > > > #5 -
              > > >
              > >
              >
              http://www.usi.edu/science/Dean/dean/CollegeSeminarSeries.asp
              > > >
              > > > #5 is the broken link.
              > > >
              > > > Those are just a couple of examples that I could
              > > > quickly come up with.
              > > >
              > > > Thanks again,
              > > >
              > > > Mike
              > > >
              > > >
              > > > --- Tilman Hausherr <tilman@s...> wrote:
              > > >
              > > > > On Wed, 5 Oct 2005 06:04:40 -0700 (PDT),
              > Michael
              > > > > Hunsaker wrote:
              > > > >
              > > > > >Sure, some of the broken links are not "real"
              > > > > broken
              > > > > >links but a good chunk of them are truly
              > broken
              > > > > links.
              > > > > > Why isn't Xenu picking these up?
              > > > >
              > > > > Tell me a link that isn't spidered, but it
              > linked
              > > > > from a page that is.
              > > > >
              > > > >
              > > >
              > > >
              > > >
              > > >
              > > >
              > > >
              > > > _____
              > > >
              > > > YAHOO! GROUPS LINKS
              > > >
              > > >
              > > >
              > > > * Visit your group "xenu-usergroup
              > > > <http://groups.yahoo.com/group/xenu-usergroup> "
              > on
              > > > the web.
              > > >
              > > > * To unsubscribe from this group, send an email
              > to:
              > > > xenu-usergroup-unsubscribe@yahoogroups.com
              > > >
              > >
              > <mailto:xenu-usergroup-unsubscribe@yahoogroups.com?
              > subject=Unsubscribe>
              > > >
              > > >
              > > > * Your use of Yahoo! Groups is subject to the
              > > > Yahoo!
              > > > <http://docs.yahoo.com/info/terms/> Terms of
              > > > Service.
              > > >
              > > >
              > > >
              > > > _____
              > > >
              > > >
              > > >
              > > > [Non-text portions of this message have been
              > > > removed]
              > > >
              > > >
              > >
              >
              >
              >
              >
              >
            Your message has been successfully submitted and would be delivered to recipients shortly.