Exluding a specific IP from banning

[Joe Belmaati]

I am using a bunch of rewrite rules in my htaccess file that will send bad bots and other menaces to a trap script, and I am wondering whether there is a way to exclude a specific IP from being logged in the SetEnvInf directive - no matter what they do...?

Any help is much appreciated.

Sincerely
Joe Belmaati
Copenhagen Denmark

Here's my htaccess file

SetEnvIf Request_URI "^(/403.*\.htm¦/robots\.txt)$" allowsome

<Files *>
order deny,allow
deny from env=getout
allow from env=allowsome
</Files>

<Files ~ "^\.ht">
Order allow,deny
Deny from all
Satisfy All
</Files>

Options +FollowSymLinks
RewriteEngine on
RewriteBase /
# Various bots
RewriteCond %{HTTP_USER_AGENT} ^WinHttp\.WinHttpRequest\.\d+ [NC,OR]
# Address harvesters
RewriteCond %{HTTP_USER_AGENT} ^(autoemailspider¦ExtractorPro) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^E?Mail.?(Collect¦Harvest¦Magnet¦Reaper¦Siphon¦Sweeper¦Wolf) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} (DTS.?Agent¦Email.?Extrac) [NC,OR]
RewriteCond %{HTTP_REFERER} iaea\.org [NC,OR]
# Download managers
RewriteCond %{HTTP_USER_AGENT} ^(Alligator¦DA.?[0-9]¦DC\-Sakura¦Download.?(Demon¦Express¦Master¦Wonder)¦FileHound) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(Flash¦Leech)Get [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(Fresh¦Lightning¦Mass¦Real¦Smart¦Speed¦Star).?Download(er)? [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(Gamespy¦Go!Zilla¦iGetter¦JetCar¦Net(Ants¦Pumper)¦SiteSnagger¦Teleport.?Pro¦WebReaper) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(My)?GetRight [NC,OR]
# Image-grabbers
RewriteCond %{HTTP_USER_AGENT} ^(AcoiRobot¦FlickBot¦webcollage) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(Express¦Mister¦Web).?(Web¦Pix¦Image).?(Pictures¦Collector)? [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^Image.?(fetch¦Stripper¦Sucker) [NC,OR]
# "Gray-hats"
RewriteCond %{HTTP_USER_AGENT} ^(Atomz¦BlackWidow¦BlogBot¦EasyDL¦Marketwave¦Sqworm¦SurveyBot¦Webclipping\.com) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} (girafa\.com¦gossamer\-threads\.com¦grub\-client¦Netcraft¦Nutch) [NC,OR]
# Site-grabbers
RewriteCond %{HTTP_USER_AGENT} ^(eCatch¦(Get¦Super)Bot¦Kapere¦HTTrack¦JOC¦Offline¦UtilMind¦Xaldon) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^Web.?(Auto¦Cop¦dup¦Fetch¦Filter¦Gather¦Go¦Leach¦Mine¦Mirror¦Pix¦QL¦RACE¦Sauger) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^Web.?(site.?(eXtractor¦Quester)¦Snake¦ster¦Strip¦Suck¦vac¦walk¦Whacker¦ZIP) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} WebCapture [NC,OR]
# Tools
RewriteCond %{HTTP_USER_AGENT} ^(curl¦Dart.?Communications¦Enfish¦htdig¦Java¦larbin) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} (FrontPage¦Indy.?Library¦RPT\-HTTPClient) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(libwww¦lwp¦PHP¦Python¦www\.thatrobotsite\.com¦webbandit¦Wget¦Zeus) [NC,OR]
# Unknown
RewriteCond %{HTTP_USER_AGENT} ^(Crawl_Application¦Lachesis¦Nutscrape) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^[CDEFPRS](Browse¦Eval¦Surf) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(Demo¦Full.?Web¦Lite¦Production¦Franklin¦Missauga¦Missigua).?(Bot¦Locat) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} (efp@gmx\.net¦hhjhj@yahoo\.com¦lerly\.net¦mapfeatures\.net¦metacarta\.com) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(Industry¦Internet¦IUFW¦Lincoln¦Missouri¦Program).?(Program¦Explore¦Web¦State¦College¦Shareware) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(Mac¦Ram¦Educate¦WEP).?(Finder¦Search) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(Moz+illa¦MSIE).?[0-9]?.?[0-9]?[0-9]?$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^Mozilla/[0-9]\.[0-9][0-9]?.\(compatible[\)\ ] [NC,OR]
RewriteCond %{HTTP_USER_AGENT} NaverRobot [NC,OR]
# Frontpage Office etc
RewriteCond %{REQUEST_URI} ^/(MSOffice¦_vti) [NC,OR]
#RewriteCond .* - [F]
# Email
RewriteCond %{REQUEST_URI} (mail.?form¦form¦form.?mail¦mail¦mailto)\.(cgi¦exe¦pl)$ [NC,OR]
# Various
RewriteCond %{REQUEST_URI} ^/(bin/¦cgi/¦cgi\-local/¦sumthin) [NC,OR]
RewriteCond %{THE_REQUEST} ^GET\ /?http [NC,OR]
# Forbid if UA is a single word - case-insensitive, A-Z only
RewriteCond %{HTTP_USER_AGENT} ^[a-z]+$ [NC]
RewriteCond %{REQUEST_URI}!/getout\.php$
RewriteRule .* /getout.php [L]
#
# Forbid if blank (or "-") Referer *and* UA, except for HEAD requests from caching proxies (such as AOL)
RewriteCond %{REQUEST_METHOD}!^HEAD$
RewriteCond %{HTTP_REFERER} ^-?$
RewriteCond %{HTTP_USER_AGENT} ^-?$
RewriteCond %{REQUEST_URI}!/getout\.php$
RewriteRule .* /getout.php [L]

[jdMorgan]

Your code needs some changes to make it work properly. The main problem is that only one "Order" statement can be present in .htaccess -- the others will be ignored.

The easiest solution to fix this is to move the logic that protects your .htaccess and .htpasswd file from the mod_access section to the mod_rewrite section. So, remove the code sections that reads:

<Files ~ "^\.ht">
Order allow,deny
Deny from all
Satisfy All
</Files>

and add the following code below "RewriteEngine on"

RewriteRule ^\.ht - [F]

Next, you'll need to allow for your "special" IP address in both the mod_access and mod_rewrite code sections. For mod_access, modify the code as shown:

SetEnvIf Request_URI "^\.ht" getout
# IP address you want to allow
SetEnvIf Remote_Addr "^192\.180\.0\.1$" allowsome
#
<Files *>
order deny,allow
deny from env=getout
allow from env=allowsome
Satisfy all
</Files>

And for mod_rewrite, place these lines directly after the .htaccess-protecting RewriteRule discussed above.

RewriteCond %{REMOTE_ADDR} ^192\.168\.0\.1$
RewriteRule .* - [L]

Change the IP address in both lines of code to relect the IP address you want to allow, using a backslash before each literal period in the IP address as shown.

Jim

[Joe Belmaati]

Thank you so much, Jim. I will give it a whirl :-)

[Joe Belmaati]

Jim,
those changes work perfectly! Thank you very much.
Sincerely
Joe Belmaati
Copenhagen Denmark