Does blocking "stealth" Microsoft crawlers impact search results?
3:39 pm on Apr 30, 2012 (gmt 0)
One of my sites gets hit thousands of times per day from Microsoft IPs with UAs not identifying as Microsoft crawlers (bingbot, msnbot, etc.). Has anyone blocked these, and if so, has it impacted search results?
4:45 pm on Apr 30, 2012 (gmt 0)
I block all plainclothes MSIEbots for the rock-solid reason that they don't honor (or even read) robots.txt.
Has it affected search results? Hm, I don't get a lot of bing traffic-- but who does?