Loading ...
Sorry, an error occurred while loading the content.

Re: My URL will not work

Expand Messages
  • Chris
    ... You ve got an error in your .htaccess file which is forcing a request for your main index.html/.htm etc to be forwarded to your
    Message 1 of 4 , Jul 22, 2009
      --- In xenu-usergroup@yahoogroups.com, "Deno Tufares, Jr."<deno@...> wrote:
      > Xenu will not search my site - gives me a "Not Found" error...
      > http://www.adtenterprises.com
      > This does not work either...
      > http://www.adtenterprises.com/cgi-bin/store/
      > It starts then gets to http://www.adtenterprises.com/cgi-bin/store/commerce.cgi and then fails with the "Not Found" error...
      > I am able to use Xenu with other sites and other commerce.cgi sites with no problem - but my site does not work. I have even tried it from a different computer at a different location with the same results. I have notified the technical people for the web hosting server and they state that there are no blocks on the server. My audit logs show that "4 visitors use Xenu Link Sleuth 1.3c as a web browser".
      > I have read through all the notes and searched this user group with no idea as to what to do. My site has a ".htaccess" file REWRITE for my site to go directly to the CGI Shopping Cart - something like:
      > RewriteEngine on
      > RewriteCond %{REQUEST_FILENAME} !-f
      > RewriteRule ^.*(\.html$|\.htm$) /cgi-bin/store/commerce.cgi
      > RewriteRule ^index\.html$ /cgi-bin/store/commerce.cgi

      You've got an error in your .htaccess file which is forcing a request for your main index.html/.htm etc to be forwarded to your /cgi-bin/store/commerce.cgi

      The problem is, rather than return the code 200 AND the content, it returns the content and the 404 error code.

      This is the header that is being returned to a GET (shown first)

      GET / HTTP/1.1
      Host: www.adtenterprises.com
      User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-GB; rv: Gecko/20090715 Firefox/3.5.1
      Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5,application/x-tsmxml
      Accept-Language: en-gb,en;q=0.5
      Accept-Encoding: gzip,deflate
      Accept-Charset: UTF-8,*
      Keep-Alive: 300
      Connection: keep-alive

      Response from your server

      HTTP/1.x 404 Not Found
      Date: Wed, 22 Jul 2009 21:40:16 GMT
      Content-Type: text/html; charset=ISO-8859-1
      Connection: keep-alive
      Server: Apache
      Set-Cookie: cart_id=1248298817.419; expires=Wed, 22-Jul-2010 21:40:16 GMT; path=/cgi-bin/store; domain=www.adtenterprises.com;
      Content-Length: 18461

      I can't remember the correct .htaccess redirect method, but if I were you I'd go searching around.

      Any visitor will see a 404, but worse still ALL search engines and spiders will also see the 404 error code.
      So despite you web server sending out the contents of /cgi-bin/store/commerce.cgi, I would hazard a guess that they all see the 404 and leave assuming that the page was not found...
      (Some search engines might crawl the 'supposed' 404 looking for other pages, but will assume that the page at www.adtenterprises.com does not exist.

      But, even worse, if I then do a

      GET /cgi-bin/store/0/12_Volt_Travel_Blanket.htm HTTP/1.1

      (looking for http://www.adtenterprises.com/cgi-bin/store/0/12_Volt_Travel_Blanket.htm)

      Your web server still sends out a 404

      HTTP/1.x 404 Not Found

      Even though the 404 is returned, the web server still sends out the content...

      If I can remember the .htaccess stuff, I'll come back and let you know.


    Your message has been successfully submitted and would be delivered to recipients shortly.