Forum Moderators: mack
For webmasters, it’s important to be familiar with how the Bing crawler interacts with your site. After the full algorithmic transition is complete, you only need to optimize for one crawler (Bing), as we will provide Yahoo! with results from our index.
During this testing phase, it’s important to look at how your site is showing up in search results on Yahoo! and Bing today, so you can assess if there are any tweaks you need to make or feedback you need to share with the team to make sure your site is accurately represented in Bing’s results.
In particular, now is a good time for you to review your crawl policies in your robots.txt and ensure that you have identical polices for the msnbot/Bingbot and Yahoo’s bots. Just to note, you should not see an increase in bingbot traffic as a result of the transition.
The selfless me knows this is not good for search.
...it’s important to look at how your site is showing up in search results on Yahoo! and Bing today, so you can assess if there are any tweaks you need to make ...
And exactly how is finally a contender to Google not good?
maybe just maybe stop their arrogant attitude towards searchers and webmasters alike.
the article refers to "Bingbot". I don't recall ever seeing a crawl by Bingbot, just msnbot. Am I missing something?
On October 1st, 2010, we will drop the beta designation from the Bing crawler and change the name of the crawler to reflect Microsoft's new brand for search. Instead of the old msnbot 2.0b showing up in your server logs, the updated user agent will be:
Mozilla/5.0 (compatible; bingbot/2.0 +http://www.bing.com/bingbot.htm)
The HTTP header From field will also change as shown below:
From: msnbot(at)microsoft.com
will become
From: bingbot(at)microsoft.com
Robots Exclusion Protocol directives precedence
We want webmasters to know that bingbot will still honor robots.txt directives written for msnbot, so no change is required to your robots.txt file(s).
I'd say the defeat of Yahoo is a major victory for Google. With Yahoo out of the way, they can concentrate one player - Bing.