Ban "Opera/9.80 (Windows NT 5.1; U; en) Presto/2.7.62 Version/11.01"

scrapers keep crashing my server

[walkman]

This is the tenth time that scrapers with "Opera/9.80 (Windows NT 5.1; U; en) Presto/2.7.62 Version/11.01" keep crashing my server by pulling 50+ pages a second. I think it's an Opera mini-server of sort.

How would I go about banning it without banning the entire Opera users?

Searching I found this, any suggestion in modifying it?

RewriteCond %{HTTP_USER_AGENT} ^Mozilla/5.*Firefox* [OR]
RewriteCond %{HTTP_USER_AGENT} ^Opera/*
RewriteRule ^/$ /index.notsupported.php [L

Every time my server has crashed I have seen the Opera ...Presto signature and I've had enough.

Thanks,

[Brett_Tabke]

yours will ban all ff and opera users.

Why not go after the IP instead? Most users can get gotten with the block of a c block...

Otherwise - season to taste:

deny from 182.131.84.59 178.49.75.201 66.235.124.59

RewriteCond %{HTTP:X-moz} ^prefetch [NC,OR]
RewriteCond %{HTTP_X_FORWARDED_HOST} ^world.altavista\.com [NC,OR]
RewriteCond %{HTTP_X_FORWARDED_SERVER} ^altavista\.com [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^libwww [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^Baiduspider [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^W3C_Validator [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^ColdFusion [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^PycURL [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^PostRank [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^MFE_expand [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^JS-Kit [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^Python [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^RockMelt [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^Missigua
RewriteRule ^.* - [F]
RewriteCond %{HTTP_REFERER} ^http://www\.iaea\.org$ [NC]
RewriteRule !^http://[^/.]\.webmasterworld\.com\.* - [F]

[wilderness]

You've not provided if the "server crasher" was the same IP range?
Generally speaking in these kinds of requests for assistance, assured best results are provided with a full raw log visitor line

# IF UA contains Opera and comes from IP range
RewriteCond %{HTTP_USER_AGENT} Opera
RewriteCond %{REMOTE_ADDR} ^123\.456\.789\.
RewriteRule .*$ - [L]

#last line with custom 403 page
RewriteRule ^(403.\shtml|robots\.txt)$ - [L]

[walkman]

Brett,
I get them from all places. For example I banned China and now it's better but then got one from NYC, a FIOS account. The other day it was a Century One DSL. Today one from a US based server (blocked their entire c Block via Iptables).

By the time I found out it's an attack everything is frozen, I can barely FTP or telnet. Usually I just change the root /home for the domain until the server recovers, it's that bad. It's that particular site, I think it's on a scraper's list somewhere and many hide their IPs so I have to do some searching. This of course, only when I happen to notice it.

But you are right, I went to a site of mine on Opera and the string is identical, I thought that this was a server string but it'd the plain Opera one.

I have to find a script that has some sort of flood control: if you get more than x files a minute, something is wrong so stop it.

[wilderness]

mod_throttle [google.com] may be worth exploring.