Page is a not externally linkable
- Search Engines
-- Sitemaps, Meta Data, and robots.txt
---- Incorrect Robots.txt URL in server Logs


triggerfinger - 4:59 pm on Sep 17, 2012 (gmt 0)


Hey Guys,

I'm seeing some seriously strange stuff in our log files.

After receiving warning in GMT about robots.txt inaccessible, we checked the server logs and are seeing the following:

66.249.73.200 www.example.com - [16/Sep/2012:12:21:54 -0400] "GET /exampleproducts/product-2012.htmlrobots.txt HTTP/1.1" 301 - "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" "-"

Any idea why Google would request incorrect URLs like this? Anyone seeing anything similar?

Thanks,

-t


Thread source:: http://www.webmasterworld.com/robots_txt/4496320.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com