Page is a not externally linkable
Staffa - 10:02 pm on Aug 17, 2011 (gmt 0)
It's about a new site. The robots.txt file is empty except for Googlebot disallowed everything.
Every bot that visits reads the file (or not, as may be the case, but that's my purpose) then happily crawls the content of the site.
Bingbot visits 7-10 times a day, reads the robots file and walks away. In nearly two weeks of passing by it has not yet taken any page from the site.
An identity crisis ;o)