Rewrite Rule

[toolman]

RewriteEngine on
RewriteCond %{HTTP_USER_AGENT} ^EmailSiphon [OR]
RewriteCond %{HTTP_USER_AGENT} ^EmailWolf [OR]
RewriteCond %{HTTP_USER_AGENT} ^ExtractorPro [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mozilla.*NEWT [OR]
RewriteCond %{HTTP_USER_AGENT} ^Crescent [OR]
RewriteCond %{HTTP_USER_AGENT} ^CherryPicker [OR]
RewriteCond %{HTTP_USER_AGENT} ^[Ww]eb[Bb]andit [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebEMailExtrac.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^NICErsPRO [OR]
RewriteCond %{HTTP_USER_AGENT} ^Telesoft [OR]
RewriteCond %{HTTP_USER_AGENT} ^Zeus.*Webster [OR]
RewriteCond %{HTTP_USER_AGENT} ^Microsoft.URL [OR]
RewriteCond %{HTTP_USER_AGENT} ^Wget [OR]
RewriteCond %{HTTP_USER_AGENT} ^EmailCollector
RewriteRule ^.*$ /badspammer.html [L]

I got this from here. When I test it with Sam Spade I get:
10/15/01 21:46:22 Browsing [XXXXX.com...]
Fetching [XXXXXX.com...] ...
GET / HTTP/1.1

Host: XXXXXXXXX.com

Connection: close

User-Agent: Wget/1.4.5

Is this right?

[Air]

Yeah, that looks right.

[toolman]

It's not supposed to go to the badspammer.html page?

[littleman]

Try this:
RewriteEngine on
RewriteBase /
RewriteCond %{HTTP_USER_AGENT} ^EmailSiphon [OR]
RewriteCond %{HTTP_USER_AGENT} ^EmailWolf [OR]
RewriteCond %{HTTP_USER_AGENT} ^ExtractorPro [OR]
RewriteCond %{HTTP_USER_AGENT} ^Mozilla.*NEWT [OR]
RewriteCond %{HTTP_USER_AGENT} ^Crescent [OR]
RewriteCond %{HTTP_USER_AGENT} ^CherryPicker [OR]
RewriteCond %{HTTP_USER_AGENT} ^[Ww]eb[Bb]andit [OR]
RewriteCond %{HTTP_USER_AGENT} ^WebEMailExtrac.* [OR]
RewriteCond %{HTTP_USER_AGENT} ^NICErsPRO [OR]
RewriteCond %{HTTP_USER_AGENT} ^Telesoft [OR]
RewriteCond %{HTTP_USER_AGENT} ^Zeus.*Webster [OR]
RewriteCond %{HTTP_USER_AGENT} ^Microsoft.URL [OR]
RewriteCond %{HTTP_USER_AGENT} ^Wget [OR]
RewriteCond %{HTTP_USER_AGENT} ^EmailCollector
RewriteRule ^(.*) badspammer.html [L]

[Air]

>It's not supposed to go to the badspammer.html page?

Yes, if you change this:

RewriteRule ^.*$ /badspammer.html [L]

to this:

RewriteRule ^.*$ badspammer.html [L]

Pretty much what Littleman said :)

[toolman]

Yes.Yes.Yes. This has to be the simplest way I've seen to fight the bot war.

[sarkye]

this looks very cool and I haven't seen anything about it from older threads (which is where I gues it came from )

Can someone tell me where these lines go? In the headers?

[theperlyking]

How about saving a little typing, e.g

RewriteCond %{HTTP_USER_AGENT} ^(Mozilla\/4\.0\ \(compatible\;\ BorderManager¦Mozilla\/3\.01\ \(compatible\;\)¦Ditto¦Email¦Bulk¦Extra¦Tele¦Site¦Extract¦Scann¦HTTPConnect¦DDemo¦EBrowse) [NC]
RewriteRule .* - [F]

This is what I have to forbid naughty UA's.

sarkye:
The rewrite facilities mentioned here are a feature of mod_rewrite, a module which can optionally be used if you use the apache web server (commonly used on unix/linux servers). The rules above go in the .htaccess file in the appropriate directory.

(edited by: theperlyking at 11:34 pm (gmt) on Oct. 19, 2001

[ggrot]

Forbid them! Don't do that. Send them to a little script that just dumps a really long (infinitely) list of randonly generated mailto urls at it. Crash the sucker.

[theperlyking]

Yeah, theres been long discussions about that. Personally I would rather just stop them dead instead of spending time on them.

Froggyman wrote a nice bot-choking script though I believe, if you want to go in that direction.

[sarkye]

Thanks so much!

I already have this up and running and it's awesome.

I just found this site less than 2 hours ago and I learned something great and picked up a new tool too.

Thank you :)

now if someone can just help with my php promblem ;)

[sugarkane]

> now if someone can just help with my php promblem

Hehe, just post your problem and I'm sure that someone around here will be able to help :)

[sarkye]

thanks sugarkane...did already...fingers crossed!