14 megs worth of hits to forbidden page?!

Help ! What the Duck?!

[walrus]

If you add an ISP to your block list, and say it happens to be a ISP like say,... adelphia.net,
should it ping you for 6 hours straight , i mean like 1000s of requests being denied and redirected to my forbid page(which has a nice little singing frog on it). Is that normal when you deny service providers?

The other day logs showed strange activity from this IP range which led me to block em.
Could anyone with experience similar to this please advise me, much appreciated.

[walrus]

I can count up to 19 requests per second. Should i expect the same behaviour from AOL or Sympatico if I was to block there users for whatever reason?

68-232-137-10.chvlva.adelphia.net - - [04/Mar/2006:15:25:03 -0500] "GET /errors/forbid.html HTTP/1.1" 302 238 "-" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 1.0.3705)"
68

[Pfui]

Occasionally I'll see relentless hits from blocked ISPs, but depending on how they're blocked, the 'reaction' is not a huge problem, and never on the level you describe.

How are you blocking this one? I don't know what you mean by "block list" -- how you're blocking either host names or IP addresses -- and immediate solutions depend on what's available to you, server-wise.

I'd use .htaccess with mod_rewrite (if it's available on your server) and [F] the specific host into oblivion. A 403-Forbidden will stop them in their tracks and stop your gushing bandwidth serving up (denying/redirecting to) a page+graphic over and over again.

The Apache Web Server Forum [webmasterworld.com] has tons of info about using mod_rewrite (which takes some head-banging but works like a charm once you get it working:) Jim Morgan knows his stuff and his posts are well worth reading/heeding.

There are other .htaccess-based solutions, too. Again, what you can do depends on your server and your access to its various capabilities.

Last but not least, in those instances when some server is maniacally out of control, I'll ask my immediate upstream provider to place a filter (read: nuke) them in their firewall(s). Plus I'll formally complain to the other host, usually via Web, sometimes via e-mail (include a section of your log file), or even phone.

Good luck!

[Staffa]

It could also be that your unwanted visitor is running in circles between the page it accesses and your 'not welcome' page.

I have seen the same thing when I started blocking, then I removed the 'not welcome' page but left the redirect to it, the visitor got a 404 not found and moved on.

[py9jmas]

Are you actually returning a 403 access denied status code? The log entry you've posted shows the status code as 302 (temporary redirect), which isn't even one of the error codes.

You can't expect bots to treat things as access denied if that isn't what they're being told.

[walrus]

Thanks for the replies, ive been using something like this:

ErrorDocument 400 www.widgets.com/errors/badrequest.html
ErrorDocument 401 www.widgets.com/errors/authreqd.html
ErrorDocument 403 www.widgets.com/errors/forbid.html
ErrorDocument 500 www.widgets.com/errors/serverr.html
CaseInsensitive On
<Files .htaccess>
deny from all
</Files>

order allow,deny
deny from somerottenbuggers.com
allow from all

I would like to use mod rewrite but changes have to be made to .htconfig It was actually Jim Morgan that advised me of that last year. Unforunately thats something the host says it cant do, even though i am on a VMS so i used the deny command as it was my only option other than changing servers.

I removed them from the deny list, and am now looking for a host and one of the requirements is that .htconf can be edited to allow for the mod rewrite.

Are you actually returning a 403 access denied status code? The log entry you've posted shows the status code as 302 (temporary redirect), which isn't even one of the error codes.
You can't expect bots to treat things as access denied if that isn't what they're being told.

Yikes! I thought that was automatic, you mean i have to have a special code in the page i send them too?
How emberassing... (silly grin)

[keyplyr]

you mean i have to have a special code in the page i send them too?

No but you need to allow them to access that forbid.html page.

Your high number are most likely the result of a loop since you are denying access but serving your own forbid.html to them at the same time. Try this if you can:

SetEnvIf Remote_Host somerottenbuggers\.com ban
SetEnvIf Request_URI ^errors/forbid\.html$ allowit
<Files *> 
Order Deny,Allow 
Deny from env=ban 
Allow from env=allowit 
</Files>

[walrus]

Thanks very much Keyplyr, I kind of suspected that, a lot of this is still way over my head, ive been using the computer for 3 years but only part time and have had so many setbacks i havnt learned half of what i'd have liked to. I mean i really thought I'd be a full time adsenser by now but....(sigh)

Thanks for the code, i've been busy with other things(drywalling,jackhammers,carpentry)to pay the rent and hav'nt had time to try it.
I hope this works with mod rewrite off because my host will not set it to allow, kinda bogus cause they are one of the largest here in Canada.

[keyplyr]

Walrus, I understood you to say it was a host, but if the code doesn't work for you try blocking the referrer:

SetEnvIf Referrer somerottenbuggers\.com ban
SetEnvIf Request_URI ^errors/forbid\.html$ allowit
<Files *>
Order Deny,Allow
Deny from env=ban
Allow from env=allowit
</Files>

[walrus]

I will be trying that, thanks again and very much appreciated. Pardon the Adsenser phrase, people must just dread that, don't know where i picked it up. :)

[jdMorgan]

py9jmas wrote:

Are you actually returning a 403 access denied status code? The log entry you've posted shows the status code as 302 (temporary redirect), which isn't even one of the error codes.

And here's the reason -- one of the most common errors in Apache configuration:

ErrorDocument 400 www.widgets.com/errors/badrequest.html
ErrorDocument 401 www.widgets.com/errors/authreqd.html
ErrorDocument 403 www.widgets.com/errors/forbid.html
ErrorDocument 500 www.widgets.com/errors/serverr.html 

The Apache documentation on ErrorDocument [httpd.apache.org] clearly states that if a canonical URL is used, the result will be a 302 redirect, and not the desired error response. The correct syntax is:

ErrorDocument 400 /errors/badrequest.html
ErrorDocument 401 /errors/authreqd.html
ErrorDocument 403 /errors/forbid.html
ErrorDocument 500 /errors/serverr.html 

With the original code posted above, you are not denying access to these unwelcome visitors, you're redirecting them.

There's another important warning about Internet Explorer's behaviour in that documentation as well.

Jim