Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- Stale bad bot lists
motorhaven - 9:39 pm on Mar 24, 2012 (gmt 0)Thread source:: http://www.webmasterworld.com/search_engine_spiders/4432835.htm
Does anyone have a decent whitelist or know of one on this site? I spend way too much time chasing down yet another variety of bad bot, scraper, etc. If the wrong people seeing it is an issue, can someone stickymail me?
Brought to you by WebmasterWorld: http://www.webmasterworld.com