Staffa - 11:49 pm on Aug 18, 2011 (gmt 0)

Tangor, I don't doubt for a moment that you either have or know of sites which are "crawled aggressively" by Bingbot but just like flashdash states on the previous page, I only see the bot come by once in a blue moon and maybe fetch a page once in a yellow moon on any of my sites. Yet none have anything specific to discourage its visits.

The site we are talking about here, well you have seen the robots.txt and the pages are as vanilla as they come, though Bbot doesn't know that yet ;o)

Another site for instance has some 3600 pages. In the year that it has been online google has crawled the whole site many times over. However, I would be surprised if Bing, in the same time span, has managed to fetch as much as 10% of the total pages only once.

As for something else to look into, I don't run anyone's ads, nor anyone's analytics, nor anyone's whatever, therefore nothing to vex any bot. So there is little to look into. Nevertheless, I appreciate your suggestion.