Auto-IP deny with cache flush

A wise move? How would you do it?

[Angonasec]

I currently check my access logs daily, and manually block IPs I find up to mischief using htaccess thus;

<Files *>
order deny,allow
# China
deny from 14. 27.8.0.0/13 etc....
</Files>

That entails a delay of 24 hrs, exposing the site to nasties until I wake up and check.

I was wondering if you think it wise to try to automate this IP deny process, and if so how would you go about it?

My concern is that any mechanism would involve auto-writing to my htaccess file, and we all know how a slight error invokes a 500, bringing the site down, so the process would have to be dependable and bomb-proof. The other concern is giving my site's password to a script to enable it to re-write my htaccess.

I know some of you do this with confidence in various bot-traps posted here at WebmasterWorld, but I'm still wary of the complicated bot-trap methods.

The main IPs I want to auto-block are those visitors who try copying to email or MS Frontpage.

For example, I currently block some email copiers and MS Frontpage using:

RewriteCond %{HTTP_REFERER} compose [NC,OR]
RewriteCond %{REQUEST_METHOD} ^(OPTIONS|TRACE|DELETE|TRACK) [NC]
RewriteRule ^(.*)$ - [F,L]

This blocks them, but I see in my logs they try repeatedly to save the page, and of course it is still in their cache.

So, I'd like to auto-block them +on IP as soon as+ they invoke OPTIONS or compose, AND somehow cause their cache to refresh so it flushes out the full-text page, and is replaced by the 403 Forbidden page they received when they were auto-blocked.

I'd prefer a non-php method, unless that is the ideal route.

Btw: Shared server, so no access to server config file.

Good or bad idea, too risky? Your thoughts please.

[wilderness]

RewriteCond %{REQUEST_METHOD} ^(OPTIONS|TRACE|DELETE|TRACK) [NC]

Overkill.

The purpose of the <Limit> directive [httpd.apache.org] is to restrict the effect of the access controls to the nominated HTTP methods. For all other methods, the access restrictions that are enclosed in the <Limit> bracket will have no effect. The following example applies the access control only to the methods POST, PUT, and DELETE, leaving all other methods unprotected:

There's an old thread [webmasterworld.com] (includes other topics as well) which explains this method.


BTW these issues are best in the SSID Forum [webmasterworld.com]

[jdMorgan]

If a page has been cached on the client side, there's nothing you can do to "un-cache" it. I'd suggest putting a very short expiry time and a "Must-revalidate" cache-control header on your "valuable" pages, and letting the client check with your server to see if the page has been modified since it was cached. Under normal circumstances, return a 304-Not Modified response. But if you've banned the IP due to unauthorized HTTP Method use, then return a 200-OK on those pages, with some 'fake' content to over-write their cached copy.

There are probably some kludgey ways to do this without scripting -- likely involving rewriting unwelcome requests to a 'fake-page-replacement' directory. But using a script would probably be a lot easier...

Having gotten into that, using one of the scripts published here to just ban abusive IP addresses from the start may seem simpler/more appealing.

Jim

[Angonasec]

Thanks Chaps!

Jim said:
Q/
If a page has been cached on the client side, there's nothing you can do to "un-cache" it. I'd suggest putting a very short expiry time and a "Must-revalidate" cache-control header on your "valuable" pages, and letting the client check with your server to see if the page has been modified since it was cached. Under normal circumstances, return a 304-Not Modified response. But if you've banned the IP due to unauthorized HTTP Method use, then return a 200-OK on those pages, with some 'fake' content to over-write their cached copy.
/Q

How about; instead of delivering a 403 when OPTIONS is used; RewriteRule such calls to a static "fake" content page, and THEN auto-block the IP in htaccess?

That way, they only ever cache the "fake" content, and their IP is auto-blocked henceforth.

Q/
Having gotten into that, using one of the scripts published here to just ban abusive IP addresses from the start may seem simpler/more appealing.
/Q

I'm very wary of the anti-bot scripts here because; they are now so old, false positives seem common, the one pixel gif looking suspicious to SEs, security, php overhead, complicated. etc.

[jdMorgan]

If you fear tweaking those scripts, then use only the part of of one those scripts that writes an IP address "ban" to the .htaccess file.

The response to an OPTIONS request is an HTTP header containing a list of options for "functions" that can be used on the page/script/resource specified by the requested URL; it is not a page of "content" like an HTML page (See the HTTP/1.1 protocol specification for details). The server directly returns only a few HTTP headers, and the content-handler phase is never invoked.

Therefore, the OPTIONS method can't be used to call a script. Actually, none of the HTTP methods listed in your original code can be used to call a script, so the method I outlined will need to be expanded. You'll have to write or get a script that notes "HTTP Method violations" but returns the expected HTTP headers, and then wait for the next GET request from that IP address, and then serve the "fake-content page" to over-write the cache and then ban the IP address. This will require code that (to my knowledge) isn't posted here.

You may want to reconsider whether making the proprietary pages available in the first place is advisable. Consider requiring a log-in and/or a cookie, and/or making the proprietary pages completely un-cacheable until a log-in is performed or a verifiable cookie is sent back with the page request.

In general, if a particular implementation gets too complicated, take a step back and look at whether that implementation actually addresses the *only* solution to the actual problem at hand. We see many posts here that are requests for implementations of the *wrong* solution to a problem. I can't tell if that might be the case here, but often, posters rush to a specific implementation before doing "systems analysis" to determine the simplest/most effective solution. g1smd and I have commented on this several times, often using the phrase, "we don't want to post the right solution to the wrong problem."

Anyway, that's something to be aware of, and it may be applicable here.

Jim