banning partial urls with .htaccess

[MsCantBWrong]

Hi. Great resource here. I love the "close-to-perfect .htaccess file" and have del.icio.us'd it.

Okay... I've been reviewing my referral logs and cross referencing my gmail trash as all comments in my photoblog that are flagged for moderation are immediately deleted and the email notification is immediately trashed.

In the past 2 days I have received 234 spam notifications. Of those 234 hits, they are spread out amongst 20 different IPs. Of those 20 IPs, 16 are from [subdomain.insertPOSspammer-inc.net....]

Each subdomain is a different drug or poker. According to ARIN, each IP is registered to a different company.

So... my question is, how can I ban [*.POSspammer-inc.net?...]

[jdMorgan]

MsCantBWrong,

Welcome to WebmasterWorld!

You can use mod_rewrite, and test the %{REMOTE_HOST} variable, looking for the substring "POSspammer". Follow this with a RewriteRule blocking access to your posting resources.

Note that testing the %{REMOTE_HOST} variable invokes a reverse-DNS lookup every time it runs, and is therefore quite inefficient. So, you want to do it as infrequently as possible, which I why I propose only restricting access to whatever resources are required to post, and not to the entire site.

Jim

[MsCantBWrong]

The url in question has already been discussed in this thread:
[webmasterworld.com...]

jdMorgan:
I'll have to do some research on that mod_rewrite. I have no idea what you said. :o)

[jdMorgan]

See our Apache forum charter (link at upper left) for links to some useful documents.

Jim

[MsCantBWrong]

Well... I've done my homework and I figure I should post my results here in case anyone else is interested in the solution.

Here is my amended .htaccess file with comments referencing where the information was found.

I *am* having a problem with the first three commented lines. Can someone take a look and maybe tell me where my problem is?

Problem: CSS file here: http://www.example.com/information/style.css

So that when: http://example.com/information/iFAQ.htm
and
http://example.com/information/100TAM.htm
are loaded, there is no style or formatting shown.

Also, when those lines are uncommented, from http://example.com/fo.tog.ra.fie/ when clicking on the ms/in.for.ma.tion link in the header... the action times out. I thought maybe it would be a problem with the partial URL block... but upon commenting those first three lines, the issue resolved itself.

In all honesty, I can live without those lines, but I have a certain animosity toward these spammers. I would like to give them a taste of their own medicine.

RewriteEngine On
# The next three lines make the spammers that squeeze past filters bounce back to their own site.
# Let them eat their own bandwidth!
#RewriteCond %{HTTP_HOST} !^example\.com$ [NC]
#RewriteCond %{REMOTE_ADDR} ^(.*)$ [NC]
#RewriteRule ^(.*)$ http://%1 [R=301,L]
RewriteCond %{REQUEST_FILENAME} .*jpg$¦.*gif$¦.*png$ [NC]
RewriteCond %{HTTP_REFERER} !^$
RewriteCond %{HTTP_REFERER} !example\.com [NC]
RewriteCond %{HTTP_REFERER} !livejournal\.com [NC]
RewriteCond %{HTTP_REFERER} !google\. [NC]
RewriteCond %{HTTP_REFERER} !search\?q=cache [NC]
#hotlink prevention - supposed to follow a hotlink to a php page and show the picture being hotlinked with a credit to my site
RewriteRule (.*) /showpic.php?pic=$1

RewriteRule ^.*_$ - [F,L]

#partial URL blocking
SetEnvIfNoCase Referer ".*(insurance ... long list of partial domain names ... viagra).*" spam_ref=yes
RewriteCond %{ENV:spam_ref} ^yes$ [NC]
RewriteCond %{HTTP_REFERER} ^(.*)$ [NC]
RewriteRule ^(.*)$ %1 [R=301,L]

# Bulgarian spammers
RewriteCond %{HTTP:VIA} ^1\.1\ symantec\ web\ security\ (2\.01\.060)
RewriteRule .* - [L,F]

# pineapple
RewriteCond %{HTTP:VIA} ^.+pinappleproxy
RewriteRule .* - [L,F]

# track back
RewriteCond %{HTTP_REFERER} ^http://12.***.72.13
RewriteRule ^(.*) /#*$!-bot

This is in addition to WizCraft's close to perfect ban list found on the following thread: [webmasterworld.com...]

Thanks for your help.
So much to learn!

[edited by: jdMorgan at 12:42 am (utc) on Feb. 1, 2005]
[edit reason] Remove specifics per TOS, fix sidescroll. [/edit]

[MsCantBWrong]

Sorry about the long list.

It's important to note that each name on the list should be separated by a pipe.

Also, I listed my urls for examples. So someone might be able to look at the code to see why it's getting flagged by the .htaccess.

Sorry.

If anyone is inclined to take a look, please message me and I will provide the URLS/source.

[MsCantBWrong]

Oh... and I left the links to the sites where I found the pertinent information because they deserve credit for their work. I did not write this .htaccess myself. It is a compilation of many people's hard work and much research.

[jdMorgan]

I'd suggest:

RewriteCond %{HTTP_HOST} !^(www\.)example\.com [NC]
RewriteCond %{REMOTE_ADDR} ^(.*)$
RewriteRule ^(.*)$ http://%1/$1 [R=301,L]

This avoids the problem of mixing references to the www- and non-www domain.

Jim

[MsCantBWrong]

Thank you for the quick reply.

That caused a 501 altogether.

I'll do some more fact finding and post results here.

Also... I should note, that since implementing the partial URL block as well as the specific UA blocks, I have received zero referral spam. So, kudos to the folks that wrote those .htaccess snippets.