Page is a not externally linkable
- Search Engines
-- Search Engine Spider and User Agent Identification
---- Thousands of Spambot IPs Hitting my Site


lucy24 - 5:43 pm on Dec 12, 2011 (gmt 0)


You can apply any RewriteRule to a single page. In fact it's good for your site, because then mod_rewrite doesn't have to slow down and evaluate the rule plus conditions for every single request it receives.

And you can throw out the <IfModule container. You're dealing with a specific site, right? So either you've got mod_rewrite or (shudder) you haven't. And, whoops, you only need to turn the RewriteEngine on once. And / is the default base so you don't need that.

Express the Referer as ^-?$ because null referers (also blank UAs) generally come through as a single -


Thread source:: http://www.webmasterworld.com/search_engine_spiders/4396669.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com