Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- Are these considered as bad bots ?
blend27 - 3:19 am on Jan 9, 2013 (gmt 0)
| some have done a better job at faking headers |
|
They still send "Keep Alive:300" :)
Thread source:: http://www.webmasterworld.com/search_engine_spiders/4533935.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com