Hammered - Over 15,000 hits from one IP in 1 hour

don't know how to handle this

[bluecorr]

Hi!

Today my site has been hammered by what I believe is a SEO (he previously checked www2 for our main keyword).

Anyway my detailed stats show a certain IP visited 538 pages and spent a staggering 12 hours on the site (UA Netscape 3.01) while the raw logs show over 15,000 hits from another IP (roughly at the same time but not identical) the vast majority with the user agent "Googlebot" (just the word) some with regular UA. The overwhelming part of the hits were like this

xxx.xx.xxx.xx - - [23/Jun/2003:13:03:41 +0100] "GET /articles/xxx/images/resources/images/images/.../images/menubar.gif HTTP/1.0"

Now the oddity is that the IP showing in the detailed ststs does appear for other users with different UA over time, sometimes as proxy but not once does it appear in the raw logs.

(replace the dots with another 50 occurances of images/ and this line repeated with slight variations for over 15,000 times!).

Now it's obvious this is an automated software that was used but was this hammering done on purpose or was it just a software glitch? (in just one hour he/she generated 100MB worth of bandwidth).

What am I supposed to do now? Can I use the IP somehow to report it or something? Should I ban it? Any advice is appreciated.

Thanks

[jeremy goodrich]

I would ban the IP (unless it could be a legitimate GoogleBot...) and then if you can get in touch by tracking them down, email them & let them know (politely) of the problem they caused your server.

[oilman]

what is the IP? Go ahead and post it here. Someone may know what it is and if it's one that should be banned.

[bluecorr]

The IP is definitely NOT from Googlebot (different from the 64.68.* or 216.239.*). In addition Googlebot's UA is "Googlebot/2.1 (+http://www.googlebot.com/bot.html)" as far as I know and not "Googlebot".

I did a traceroute of the IP but I really don't know how to use the info to track them down.

Thanks!

[bluecorr]

The IP showing in my raw logs (with the over 15,000 hits) is *.*.*.* The IP showing in my detailed stats (the one with the Netscape UA is *.*.*.* which also appears as a proxy for other users).

I hope posting the IPs is not against the rules. If it is I apologise.

[edited by: Brett_Tabke at 4:30 pm (utc) on June 25, 2003]
[edit reason] ip's masked at the request of the ip owner [/edit]

[oilman]

drop the IP in the ARIN whois [arin.net] tool and see what comes back. If it gives you the name of an ISP you can just write them and tell them a user his abusing your site. In the meantime go ahead and ban them if you want.

[Romeo]

I would track them later.
First I would put them into my firewall to drop their requests silently onto the floor to minimize traffic:
iptables -A INPUT -s <offending IP> -j DROP

This free "ipdrop" script works fine and has saved me in the past (I am not affiliated with that):
http://www-106.ibm.com/developerworks/linux/library/l-fw/?dwzone=linux
http://www-106.ibm.com/developerworks/linux/library/l-fw/dynfw-1.0.tar.gz

Regards,
R.

[oilman]

looks like both those ips trace back to UK based ISPs. Probably someone running a home grown spider or perhaps ripping your whole site.

[bluecorr]

Okay I've banned him with htaccess.

I doubt they are ripping the site (which has about 100 pages). Nearly all the GET were aimed at dummy urls with lots of images/ in them.

I've done whois with ARIN but the info isn't very helpful. It's obviously a bitter SEO because we jumped to the first page for our main keyword this update.

[killroy]

*.*.*.* is an address in london, part of the netAxist Service network.
Info for netAxis is:

<snip>

courtesy of Visual route.

perhaps it helps...

SN

[edited by: Brett_Tabke at 4:31 pm (utc) on June 25, 2003]
[edit reason] IP's masked at the request of the ip owner [/edit]

[bluecorr]

Thanks for the info. Somehow I didn't get such detailed info when I did the whois. Now I'm wondering if they will believe me at all...

[DaveN]

send them your logs.

Dave

[bluecorr]

That's 16MB worth of logs (generated by the hammering itself). I will send them a chunk of it and hopefully it will be convincing enough.

Thanks

[StanBo]

Wise enough. 16 MB file is never the best way to start a conversation, and an abstract of, say, 50-80 hits with a notion that it's but a small abstract of 16 MB log is almost always convincing. And don't forget to mention that you're more than pleased to send the entire lot if an abstract is not sufficient :)

[killroy]

And I do recommend you try out Visual Route. I always find it very helpfull tracking down those rouge log entries.

SN