Welcome to WebmasterWorld Guest from

Forum Moderators: Ocean10000 & incrediBILL & keyplyr

Message Too Old, No Replies


12:08 am on Jan 7, 2012 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Nov 5, 2005
posts: 2038
votes: 1

This showed up when I was testing a Google+ Profile Author link -- I think. Yet another Google oddity, particularly the second one,second one...
04:31:09 /welcome.html
04:31:10 /dir/filename.html

No robots.txt, of course. It's been a _l_o_n_g_ time since ANY of the scores of Google bots other than the official Googlebot from .googlebot.com even request it.
2:37 am on Jan 7, 2012 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
votes: 352

You could wait until the cows come home for Googlebot-Image to ask for robots.txt in its own name. But all the different googlebots talk to each other, so I think the question is whether richsnippets considers itself a robot or not. Yes, yes, OK: whether the humans who programmed it included a directive to obey robots.txt.

The name suggests it comes down on the "robots" side. (Preview and Translate don't. In the case of Translate, I've verified this by direct experiment. In the case of Preview, a loaded htaccess still beats a robots.txt file.)

Any way to test whether it obeys "nofollow"?