Leonard0 - 5:25 pm on Jul 22, 2010 (gmt 0)
the article refers to "Bingbot". I don't recall ever seeing a crawl by Bingbot, just msnbot. Am I missing something?
msnbot will change to bingbot on Oct 1
On October 1st, 2010, we will drop the beta designation from the Bing crawler and change the name of the crawler to reflect Microsoft's new brand for search. Instead of the old msnbot 2.0b showing up in your server logs, the updated user agent will be:
Mozilla/5.0 (compatible; bingbot/2.0 +http://www.bing.com/bingbot.htm)
The HTTP header From field will also change as shown below:
Robots Exclusion Protocol directives precedence
We want webmasters to know that bingbot will still honor robots.txt directives written for msnbot, so no change is required to your robots.txt file(s).