Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

Server Logs & Redirects to Googlebot

How to read and understand the cryptics?

         

dmje

3:08 am on Mar 9, 2006 (gmt 0)

10+ Year Member



Below is a portion of my server log, as you can see the same IP requested my robots.txt file and it first returned a 301 code then 5 seconds later it requested it again and got the 200 code.

My question is this, could it be that the first request was directed to the non www site and since we have a 301 redirect in place for non www to www could that be the second request? Seems logical that is what it would be, but I am not quite clear on how to read these server logs.

66.249.71.62[07/Mar/2006:11:02:57 GET /robots.txt HTTP/1.0 301 - Googlebot/2.1 (+http://www.google.com/bot.html)
66.249.71.62[07/Mar/2006:11:03:02 GET /robots.txt HTTP/1.0 200 - Googlebot/2.1 (+http://www.google.com/bot.html)

Dijkgraaf

9:29 am on Mar 9, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



If you have a 301 redirect in for non www requests, then yes, that is probably exactly what happened.
The server log doesn't store the domain name that was requested in the http header unfortunately, so you won't see it in your log.