Welcome to WebmasterWorld Guest from 174.129.127.214

Forum Moderators: mack

Message Too Old, No Replies

MSN spidering without proper UA

   
10:46 am on Dec 20, 2011 (gmt 0)

5+ Year Member



Hi guys,

I'm not really sure if this is regular practice with MSN (Bing) or not. Since a few weeks back, they've been spidering my website without a proper User Agent (meaning their UA does NOT say anything about bingbot. It's just a regular browser UA). This happens from three different IP ranges. Yes, I've checked thoroughly and the IP rangers are registered under MSN.

I doubt this is manual review, since it doesn't look at all as normal human behavior.

Did they officially stop declaring their bots in User Agent strings?

Thanks!
2:44 pm on Dec 20, 2011 (gmt 0)

WebmasterWorld Administrator engine is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



There are some significant discussions on the stealth of the bots.

MSN's Stealth Missions [webmasterworld.com]
MSN's many cloaked bots. Again. [webmasterworld.com]
6:57 am on Dec 21, 2011 (gmt 0)

5+ Year Member



Thanks for pointing me in the right direction.