Forum Moderators: phranque

Message Too Old, No Replies

Is there an *error* in my .htaccess ErrorDocument?

Site not getting spidered

         

roldar

7:45 am on Feb 24, 2005 (gmt 0)

10+ Year Member



I've been waiting weeks for my site to be spidered by googlebot, but every time it comes around it gets lost or bored. The first few times it just showed up, looked for my favicon [and got a 404 since I don't have one], and then left. My front page hasn't even been cached yet.

After a long absense, gb showed up again last night. I'm not sure what's wrong, but here's the transcript:

-----------------------

/robots.txt
Http Code: 200
Date: Feb 23 03:22:52
Http Version: HTTP/1.0
Size in Bytes: 23
Referer: -
Agent: Googlebot/2.1 (+http://www.g**gle.com/bot.html)

/
Http Code: 302
Date: Feb 23 03:22:53
Http Version: HTTP/1.0
Size in Bytes: 301
Referer: -
Agent: Googlebot/2.1 (+http://www.g**gle.com/bot.html)

/500error.html
Http Code: 200
Date: Feb 23 03:23:05
Http Version: HTTP/1.0
Size in Bytes: 223
Referer: -
Agent: Googlebot/2.1 (+http://www.g**gle.com/bot.html)

-----------------------------

Is there a problem in my .htaccess file? I'm afraid I don't even follow exactly what it is that went wrong.

Here's my .htaccess:

-------------

ErrorDocument 401 /401error.html
ErrorDocument 403 /403error.html
ErrorDocument 404 /404error.html
ErrorDocument 500 /500error.html

--------------

Thanks in advance for any help.

jdMorgan

1:58 pm on Feb 24, 2005 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



1) Googlebot requested /robots.txt, and decided that it was not disallowed. OK so far.

2) Googlebot requested "/" (your home page) and your server returned a 302-Moved Temporarily redirect. Not good.

3) After the 302 redirect response, Google requested the URL specified in that response, and some configuration code or a script on your site failed, producing a 500-Server error response.

Your first step should be to carefully review your server error log to find out why the server 'crashed' after the Google requeswted the new URL after the redirect. Then you should fix that problem. Next, find out why your server returns a 302 redirect for a request for your home page. That is 'highly irregular' and is cause for concern.

Jim