Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- Thousands of Spambot IPs Hitting my Site


wilderness - 10:11 pm on Dec 11, 2011 (gmt 0)


You have located a solution to stop-the-bleeding, now you should spend some time in learning to understand htaccess and how to implement methods which will prevent other types of abuses to your site (s) in the feature.

As far as the innocents on refer?
Generally speaking its a bad idea to block blank refers.
Blank UA's on the other-hand are required blocking.

1) If you wish to make exceptions for IP's on your refer rule, just add IP lines to the conditions.

RewriteCond %{HTTP:Accept-Language} ^$ [OR]
RewriteCond %{HTTP_REFERER} ^$
RewriteCond %{REMOTE_ADDR} ^123\.456\. [OR]
RewriteCond %{REMOTE_ADDR} ^456\.567\. [OR]
RewriteCond %{REMOTE_ADDR} ^567\.678\.
RewriteRule .* - [F]

1) Use the "[OR]" for multiple lines, and do NOT neglect omitting the [OR] from the last IP line.
2) Once you have this band-aid in place, than you'll need to go back and learn how to use regex for IP-Ranges as opposed to specific IP's.


Thread source:: http://www.webmasterworld.com/search_engine_spiders/4396669.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com