A Close to perfect .htaccess ban list - Part 2

[adriaant]

<modnote>
continued from [webmasterworld.com...]

---

UGH, bad typo in my original post. Here's the better version (I wasn't able to re-edit the older post?):

I'm trying to ban sites by domain name, since there are recently lots of reference spammers.

I have, for example, the rule:

RewriteCond %{HTTP_REFERER} ^http://(www\.)?.*stuff.*\.com/.*$ [NC]
RewriteRule ^.*$ - [F,L]

which should ban any sites containing the word "stuff"
www.stuff.com
www.whatkindofstuff.com
www.some-other-stuff.com

and so on.

However, it is not working, so I am sure I did not setup a proper pattern match rule. Anyone care to advise?

[edited by: jatar_k at 5:06 am (utc) on May 20, 2003]

[Wizcrafts]

Q: Why would a POST method to my FormMail spam trap not spring the trap?

My logs shows POSTs to my IP trap script, but the IP did not get self-banned; I had to do it manually. Can anybody show me a Rewrite condition to add to, before, or after this RewriteCond to include POSTs? Right now only GETs trigger the script. However, POSTs to FormMail.anything do get 403s, just not auto-banned.

RewriteCond %{REQUEST_URI} (.?mail.?form¦form¦(GM)?form.?.?mail¦.?mail)(2¦to)?\.?(asp¦cgi¦exe¦php¦pl¦pm)?$ [NC]
RewriteRule .* path_to_my_trap_script [L]

BTW: I'd like to see an example of a mod to trap.pl that sends an email when the trap is sprung, if anybody has written such an addition.

Thanx, Wiz

[jdMorgan]

Wiz,

You've got something else interfering with your call to the trap -- maybe a preceding <limit> statement that is rejecting POSTs before they even get to the trap invocation.

stapel posted the modification you're looking for last year. Try here [webmasterworld.com].

Jim

[Wizcrafts]

Jim;
I have this just ahead of the Rewrite section, which I think I got from you:

<Files *>
<LimitExcept GET POST>
deny from all
</LimitExcept>
</Files>

I also have this condition several lines down from the formmail section, which leads to my main [F] rules:

RewriteCond %{HTTP_USER_AGENT} ^Mozilla/4\.06\ \(Win95;\ I\) [OR]

Could this be the cause of POST not recording an IP on the trap, since this is the UA?
Oddly, I have second trap named formmail.pl that gives a 200 found response to POST, then does nothing but present its message, else unless they use GET.

[jdMorgan]

Wiz,

Your LimitExcept won't stop POST, because it allows GET and POST and I think you're saying the user-agent block is after the call to your script, so neither of those sound like the problem.

Some hosting services intercept formmail queries before they even get to the hosted account's level. This is often the case when the host does not allow customers to use formmail.

Jim

[Wizcrafts]

Jim;
I am able to use POST to my actual mailer script, and FormMail is allowed, but must be approved patched versions. I actually helped my host by posting information about FormMail exploits and what to do about preventing abuse. I will ask my host if he is blocking incoming POSTs. I never thought of that before.

In the meantime, here is the log of the attempt that was logged but not banned:

152.163.252.70 - - [29/Sep/2003:03:02:07 -0400] "POST /cgi-bin/FormMail.pl HTTP/1.0" 302 191 "-" "Mozilla/4.06 (Win95; I)"
152.163.252.100 - - [29/Sep/2003:03:02:07 -0400] "POST /cgi-bin/FormMail.cgi HTTP/1.0" 403 4105 "-" "Mozilla/4.06 (Win95; I)"

This was reported to abuse at aol.com.
Note that I have forwarded calls to "FormMail.pl" to be accepted by another script, which is supposed to record the IP for inclusion in the ban env. That first POST hit the trap but didn't get recorded.

[claus]

>> didn't get recorded

Mozilla/4.06 (Win95

;

 I)

That's not nice...perhaps we should look into escaping some chars in the bot-trap.

/claus

[Wizcrafts]

Claus;
What am I missing here? The trap is Key_Master's trap.pl, which only cares about IP addresses, not User Agents. I have that UA banned in my list of conditions as follows:

RewriteCond %{HTTP_USER_AGENT} ^Mozilla/4\.06\ \(Win95;\ I\) [OR]

Furthermore, the attempt coming from the same UA to POST to FormMail.cgi (non-existant) got my custom 403 page, which is definitely covered by the above rule. The problem is that the RewriteCond well ahead of this rule is supposed to divert calls to any variation of FormMail to the trap script, not the 403 page. After the IP gets banned, then they should get my 403 if they come back.

All of my pertinent codes are posted on the thread.

Wiz

[jdMorgan]

Wiz,

Well, I'm stumped, but it's very interesting that the first POST attempt in your log entry shown above gets a 302-Moved Temporarily redirect. How/why is that happening? The answer might provide a clue to your larger problem.

Typically, as you show in your rewrite code snippet, an internal (transparent) rewrite is used to pass the request to the script, and not an external 301 or 302 redirect. So, that log entry is curious.

Jim

[Wizcrafts]

Jim;

That redirect is caused by this line:

RedirectMatch cgi-bin/FormMail.pl formmail.pl

I added it because A) the all-inclusive IP ban line is not working, and B) I see many more requests to FormMail.pl, than to formmail.pl. That's why I created a 302 redirect to the lowercase filename, which is also a trap script. The log I showed shows that the redirect did work, but the IP was neither recorded nor banned.

Wiz

[claus]

>> trap is Key_Master's trap.pl, which only cares about IP addresses

My bad, sorry. The ";" is not the problem then.

/claus