Forum Moderators: phranque

Message Too Old, No Replies

Referrer spam blocking

...without blocking my own site.

         

Etruscan

5:34 pm on May 26, 2006 (gmt 0)

10+ Year Member



So I've been getting a few hundred cases of referrer spam every day. This is chewing up my bandwidth and making an absolute mess of my logs. I decided to do something about it.

RewriteEngine On
RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*(-¦.)spammer(-¦.).*$ [NC,OR]
RewriteCond %{HTTP_REFERER} (discrete-?encounters) [NC,OR]
RewriteCond %{HTTP_REFERER} (cheating-?wives) [NC,OR]
RewriteCond %{HTTP_REFERER} (housewives) [NC]
RewriteCond %{HTTP_REFERER} (sleeping) [NC,OR]
RewriteRule \.*$ - [F,L]

...at the same time, I decided to clean up my URL's using MOD_REWRITE so instead of:

http://www.example.com/article.html?id=1209

They look like:

http://www.example.com/article/Sleeping-Is-Good-For-You/

Wonderful. The problem is - when I've utilized a word in the title of the article (which now appears in the URL) and that word appears in the referrer spam list (in this case the word 'sleeping'), that page on my site gets blocked by my technique. Is there a way to write a condition that will disregard my own site from being detected as referrer spam?

jdMorgan

3:38 pm on May 27, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Add:

RewriteCond %{HTTP_REFERER} !^http://(www\.)?your-domain\.com

Jim

Etruscan

6:52 am on May 28, 2006 (gmt 0)

10+ Year Member



Ahh the infamous NOT operator - now I feel stupid... but I'm also completely grateful!

Wizcrafts

5:45 pm on May 28, 2006 (gmt 0)

10+ Year Member



Etruscan;

I also use a set of rules to block referer field spammers and have discovered that some of them are exploiting legitimate, non-stop-word, unsecured .asp pages to append the spam URL. Your rule needs to be altered to allow for the occurance of the http rule anywhere in the referer field.

RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*(-¦.)spammer(-¦.).*$ [NC,OR]

Change to remove leading anchor ^:

RewriteCond %{HTTP_REFERER} [(www\.)?.*(-¦.)spammer(-¦.).*$...] [NC,OR]

You can PM me for details about how they perform this exploit.

Wiz

jdMorgan

6:26 pm on May 28, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



This regex is redundant:
(-¦.)
It can be entirely replaced with "."

A trailing .*$ pattern is also redundant.

The whole thing could shortened to


RewriteCond %{HTTP_REFERER} http://.+spammer. [NC,OR]

or even just

RewriteCond %{HTTP_REFERER} .spammer. [NC,OR]

if the idea is to require at least one character before and after "spammer".

Jim