| Welcome to WebmasterWorld Guest from 22.214.171.124 |
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
|Become a Pro Member|
|Googlebot GET /robots.txt HTTP/1.0" 404 |
Googlebot is the only bot getting 404
I have validated the robot file numerous times and can't find anything wrong with it. Googlebot is the only bot getting the 404 error.
I have also noticed that Googlebot will request robots.txt than request a page and then request robots.txt again. It than proceeds to repeat this cycle over and over.
Any clues what is going on?
All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved