Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- Strange Inktomi Corporation IP

wilderness - 1:24 am on May 6, 2012 (gmt 0)


If their ability to persist is your only concern?
The following will stop them in their tracks:

RewriteCond %{HTTP_USER_AGENT} Linux
RewriteCond %{REMOTE_ADDR} ^74\.6\.
RewriteRule .* - [F]

Thread source:: http://www.webmasterworld.com/search_engine_spiders/4449866.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com