Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- Msn


Pfui - 6:19 pm on Mar 7, 2009 (gmt 0)


Speaking of odd things -- another MSN IP:

65.55.232.nn
Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; SV1; .NET CLR 1.1.4322)

robots.txt? YES
(But only after asking for a subdir html)

So many bots, so little time...


Thread source:: http://www.webmasterworld.com/search_engine_spiders/3863781.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com