Page is a not externally linkable
- Code, Content, and Presentation
-- Apache Web Server
---- Is it possible and advisable to block HTTP1.0 requests?


not2easy - 2:49 pm on Feb 22, 2012 (gmt 0)


OK, I think I know what I need to do now. I have a trap for bots that ignore robots.txt and it automatically adds individual IPs like this:
SetEnvIf Remote_Addr ^123\.238\.10\.71$ getout I clean it out when there is an obvious bad neighborhood and block the IP group. It sets an environment to block and allows all that are not in that environment. I have had trouble trying to block by additional rewrite rules further down the htaccess file and I think it is because of the inverse Env. set in this process. I'm thinking that I can add this referrer block into the environment that gets blocked as:
SetEnvIf Referer ^http://.\.ru/$ getout
SetEnvIf Referer ^http://.\.pl/$ getout

rather than a new rewrite rule.

I don't know how the whole thing works, but these crappy domains end up showing in my GWT account as inbound links. My stats/logs are not public. I commonly read that referrer spam cannot hurt your site but I don't think it is helping anything if non-existent links appear as links to your site and the list keeps growing.


Thread source:: http://www.webmasterworld.com/apache/4420156.htm
Brought to you by WebmasterWorld: http://www.webmasterworld.com