Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- msnbot/2 snapshots?


caribguy - 1:04 am on Sep 20, 2009 (gmt 0)


Yep 1.1 uses .gz - former, as in mentioned first...

I wouldn't dare to even wager a guess on why M$ is doing this. To me it falls in the same category as the referrer spam discussed here before, or their attempts to grab images and truncated urls with the WinHTTP user agent...

Very tempting to add yet another directive to my rewrite rules...


Thread source:: http://www.webmasterworld.com/search_engine_spiders/3992111.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com