Pfui - 10:14 pm on Sep 11, 2011 (gmt 0)

Or they're developing the equivalent of a snapshot/Google Web Preview thing? Or they could care less and less about robots.txt? ;)

Note that "Yahoo! Slurp/3.0" was a sure-fire culprit at my end and it crawled from .yst and .crawl servers.