Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- MSNBot has become a constant Fast-Scraper
lucy24 - 5:34 pm on Dec 26, 2011 (gmt 0)Thread source:: http://www.webmasterworld.com/search_engine_spiders/4401159.htm
|is it the Bingbot UA that they are crawling with |
Probably not. As noted elsewhere in this Forum, the msnbot seems to like going out in street clothes. But the IP is kinda hard to disguise.
Brought to you by WebmasterWorld: http://www.webmasterworld.com