Forum Moderators: phranque

Message Too Old, No Replies

One out of whack visitor: 19,300 hits in < 2hours

What is suggested course of action...

         

caribguy

1:36 am on Nov 28, 2008 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



While grepping my logfiles for unusual activity, I came across a visitor from <edited> who ripped through one of my image-heavy sites with the following UA: "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; .NET CLR 1.1.4322)"

To make matters worse, each request forced a 200 response: including style sheets and many other files that would normally trigger a 304. Not looking forward to my bandwidth overage charge...

Skimming the 6.4MB log this fellow single-handedly (probably literally) generated, I have to presume that he manually criss-crossed the website with his misconfigured browser. I don't think this was done through a download tool.

Rather than blocking the ip address (which might well be dynamically allocated), I would love to hear some suggestions on how to auto-throttle / temporarily block users like this one. Can it be done through Apache itself, or without incurring too much of an overall performance penalty?

[edited by: physics at 1:54 am (utc) on Nov. 29, 2008]
[edit reason] Removed IP information [/edit]

physics

3:52 am on Nov 28, 2008 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Check out the mod_dosevasive apache module

caribguy

7:43 pm on Nov 28, 2008 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Thanks physics!

The user's behavior was not quite DOS-ing, rather accessing a great number of different files in a session that lasted almost two hours...
This for example never came close to happenening:

- The IP address of the requestor and the URI are both hashed into a "key".
A lookup is performed in the listener's internal hash table to determine
if the same host has requested this page more than once within the past
1 second.

Perhaps I should have asked differently: is there a utility to limit the max amount of data transferred by any user in a particular time period (ideally both per minute and per browser session)?

Otherwise, I might roll my own using cookies - is that too naive a thought?

physics

1:26 am on Nov 29, 2008 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Oops, looks like it's called mod_evasive now.
[zdziarski.com...]

It seems like it can do what you want:


DOSSiteCount
Same as 'DOSPageCount', but corresponds to the number of requests for a given site, and uses the 'DOSSiteInterval' interval.

DOSSiteInterval
Interval for the 'DOSSiteCount' threshold in second intervals.


[theserverpages.com...]

This, and any module / solution like it, is to be used cautiously though. You don't want to end up blocking good bots or users...

Download it and have a look at the README / source before you go off writing your own though ;)