Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- possible botnet


lucy24 - 7:37 pm on Aug 20, 2012 (gmt 0)


Well, if they'd asked for anything more, that MSIE 5 would have got them blocked at the gate, wouldn't it? :) Or at least rewritten. (Mine says "I'm sorry, but the server thinks you are a robot.")

:: detour to own htaccess as I realize I forgot to exempt robots.txt from the UA-based rewrite ::


Thread source:: http://www.webmasterworld.com/search_engine_spiders/4474787.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com