Page is a not externally linkable
triggerfinger - 4:59 pm on Sep 17, 2012 (gmt 0)
Hey Guys,
I'm seeing some seriously strange stuff in our log files.
After receiving warning in GMT about robots.txt inaccessible, we checked the server logs and are seeing the following:
66.249.73.200 www.example.com - [16/Sep/2012:12:21:54 -0400] "GET /exampleproducts/product-2012.htmlrobots.txt HTTP/1.1" 301 - "-" "Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" "-"
Any idea why Google would request incorrect URLs like this? Anyone seeing anything similar?
Thanks,
-t