Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- Now seeing Bingbot


jdMorgan - 3:00 am on Oct 22, 2010 (gmt 0)


Yes, and a Staffa pointed out on the 15th, now seeing the bingbot UA fetching robots.txt. I'm looking forward to shortening/simplifying my robots.txt file (and the code that produces it), so I'm hoping they'll pull the plug on msnbot soon...

Jim


Thread source:: http://www.webmasterworld.com/search_engine_spiders/4208507.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com