Dirty Agent Tricks

How to deny

[nuhkweb]

Hi,

Can I deny, using mod_rewrite using 2 conditions?

Someone using the same IP's (3 of them) of the dial-up connections that half of our province uses (including me)
is spidering and downloading all my pages regulary.

I can not deny the IP's as they are used by half of my users.

He also regulary changes the useragent of the crawler, using the names of the most interesting bots.

this is what he/she used until now:
Scooter/3.3
Scooter/3.3_SF
Googlebot/2.1 (+http://www.googlebot.com/bot.html)
Mozilla/5.0 (Slurp/si; [e-mail]; [inktomi.com...]

So can I write a mod_rewrite rule using 2 conditions, meaning --- and IP --- and UserAgent?

Thanks,

[jdMorgan]

nuhkweb,

Block any one of three user-agents coming from any one of three IP addresses:

RewriteCond %{HTTP_USER_AGENT} (^Scooter¦^Googlebot¦Slurp)
RewriteCond %{REMOTE_ADDR} ^127\.0\.0\.1$ [OR]
RewriteCond %{REMOTE_ADDR} ^127\.0\.0\.2$ [OR]
RewriteCond %{REMOTE_ADDR} ^127\.0\.0\.3$
RewriteRule .* - [F]

Note that Slurp is not start-anchored, and the the last RewriteCond must not have an [OR] on it... and the first RewriteCond should not have one either in this specific case.

Ref: Introduction to mod_rewrite [webmasterworld.com]

HTH,
Jim

[bcc1234]

If you have multiple RewriteCond statements before RewruiteRule, they must all match (AND). Unless you specify [OR] flag.