Apache dies because of "attack"

Apache vhost flooded by GET and POST requests and dies

[Philby]

Last week we've had something strange happening: coming from somewhere inside a clients internal network, a flood of GET and POST requests flooded a webserver where their website is being hosted, among many others.

The requests were all similar to these:

000.000.000.00 - - [19/Jul/2006:16:36:10 +0200] "GET /2.0/r090.example.net/zbmbg9kqq6vsimzpisxm52p33sffyh2sr53gxh2,ConnType=KeepAlive HTTP/1.0" 404 8756 "-" "Mozilla/4.0 
(compatible; MSIE 5.5; Win32)"
000.000.000.00 - - [19/Jul/2006:16:36:10 +0200] "POST 
/2.0/r090.example.net/uta2325rcby66qnxfdyb6fuh8z6f2brg4cnyf7i,ConnType=KeepAlive HTTP/1.0" 404 8756 "-" "Mozilla/4.0 
(compatible; MSIE 5.5; Win32)"

Apache seems to have started to go down almost immediately upon receiving this kind of requests.

There are several points I'm wondering about:

- How could we better stop such an "attack" (probably someone on the client side playing around with example.net, not knowing what he/she is doing)... write a filter rule in .htaccess for ModSecurity? (It's a shared server, so I don't have access to httpd.conf)

- Why is it that Apache reacts so sensitively to these requests? From what I'm told by the server provider, a few requests like these were enough to kill Apache. There were about 6'000 such requests made over a timespan of about 4 hours -- to me (a complete know-nothing in Apache matters) this does somehow not really look *that* impressive...

TIA for any insights!

-Phil

[edited by: jatar_k at 3:07 pm (utc) on July 24, 2006]
[edit reason]
[1][edit reason] no urls thanks [/edit] [/edit][/1]

[abates]

A couple of things I can think of which aren't obvious from the log entries:

Were all the requests from different IP addresses? I can see if the connection was kept open, then they could have used up all available connections to the server...

Could also be that the POSTs included megabytes of data, which isn't obvious from the log because it only shows the number of bytes sent to the user...

[Philby]

To the webserver and the firewall, all the requests were from the same IP address - the external address of our client's internal network.

The firewall admin told me that no packets were sent ("no active connection").

Apache log files show this kind of entries:

[Wed Jul 19 12:21:53 2006] [error] [client 000.000.000.00] File does not exist: /home/httpd/vhosts/clientwebsite.net/httpdocs/2.0
[Wed Jul 19 12:22:48 2006] [error] [client 000.00.000.00] mod_security: Access denied with code 406. Error reading request body, error code 104: Connection reset by peer [uri "/2.0/r050.groove.net/uvycqdsvzcjiegntumyc3mkexrbdnifqxwi94pi,ConnType=LongLived"] [unique_id "odlOW1BKjkkAAD2OByY#*$!xO"]

The folks handling the server were so baffled that they thought the server's hardware was failing, as there simply was not enough traffic to immediately recognize what was happening.

We're probably going to try to re-enact this little episode, and set up a filter in the firewall to block this kind of traffic.

However I'm still interested to know how these requests could overload Apache so thoroughly.

[scintex]

Not sure if this helps but I know when using Perl the max limit of a filename is 39 characters. That's true of those first logs that you posted.
I'm not sure if this is also a characteristic of Apache as well or if there is a rogue Perl script in the client's network causing these problems.