allow everyone to request robots.txt just to see who's knocking
Ditto. And the same presumably faux Opera+Presto combo does a lot of robots.txt-'checking' from AWS, sometimes in groups, sometimes not.
No clue which bum's behind this bot-running:
ec2-50-16-90-19.compute-1.amazonaws.com
11/11 04:25:57 /robots.txt
ec2-50-19-183-19.compute-1.amazonaws.com
11/11 04:25:28 /robots.txt
ec2-204-236-139-34.us-west-1.compute.amazonaws.com
10/31 19:47:23 /robots.txt
ec2-107-20-50-248.compute-1.amazonaws.com
09/12 19:25:14 /robots.txt
Believe it or not, there's a lot of AWS badness I don't report in the
AWS=Bad Bots [
webmasterworld.com...] threads:) Like most of the
URI=REF [
webmasterworld.com...] hacked hits like:
ec2-50-19-40-132.compute-1.amazonaws.com
Mozilla/5.0 (Windows NT 5.1; U; en) Opera 8.01
10/31 1n:21:06 URI=REF
10/31 1n:21:10 URI=REF
10/31 1n:21:13 URI=REF
10/31 1n:21:17 URI=REF
Speaking of Opera --
Months back I finally decided to take a fairly radical step and redirect ALL Opera variations but for Opera Mini and/or hailing from Opera-specific domains and IPs, and .se (Sweden). Legit Opera users see info about changing browsers, e-mailing for help, etc. The botnet fakes eat botbait.
Speaking of Opera, Entrée Deux --
Click the following Project Honey Pot link for the Russian Federation-based academ.org (looks good; acts bad) and note the impossible, and ever-changing, O+P version numbers like:
host-85-118-226-155.academ.org [
projecthoneypot.org...]
Opera/9.491 (Windows NT 6.0; U; nl) Presto/471.911.651 Version/311.191
Very clever. Very insidious.