Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- Are these considered as bad bots ?
blend27 - 3:19 am on Jan 9, 2013 (gmt 0)Thread source:: http://www.webmasterworld.com/search_engine_spiders/4533935.htm
|some have done a better job at faking headers |
They still send "Keep Alive:300" :)
Brought to you by WebmasterWorld: http://www.webmasterworld.com