Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- MSN's many cloaked bots. Again.


dstiles - 11:21 pm on Aug 17, 2010 (gmt 0)


Don't think I've has anything quite that fast. Nearest was *&^%$ securitymetrics who bombed one of my sites for 45 minutes. They are totally IP banned now!

Re: google mistake above - just found a dozen or so hits from one of their bot IPs asking for favicon. With no UA at all AND looking in the sites' roots, where it ain't! :)


Thread source:: http://www.webmasterworld.com/search_engine_spiders/4182830.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com