Forum Moderators: open

Message Too Old, No Replies

amazonaws goes stealth

         

incrediBILL

3:48 pm on Jun 9, 2008 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



I never needed to block them amazonaws ranges or RDNS before because all their bots had names and my white listing locked them out automatically...

... until today!

They came crawling using the following UA:
"Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.1.4322)"

Now I'm installing hard blocks on them.

Here's the IP ranges involved in today's scraping:

67.202.31.* ec2-67-202-31-*.compute-1.amazonaws.com.

67.202.49.* ec2-67-202-49-*.compute-1.amazonaws.com.

67.202.50.* ec2-67-202-50-*.compute-1.amazonaws.com.

67.202.54.* ec2-67-202-54-*.compute-1.amazonaws.com.

67.202.57.* ec2-67-202-57-*.compute-1.amazonaws.com.

75.101.194.* ec2-75-101-194-*.compute-1.amazonaws.com.

More importantly, it was executing bot GET and POST which is notable because some of my navigation is in JS and uses a drop down form field just to thwart bot crawling.

Well, that ship has sailed as they appear to have executed that code a few times.

No, it wasn't human, the page accesses were too fast and high volume, about 600 pages total.

Samizdata

7:05 pm on Jun 9, 2008 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month



A scan of my logs suggests that no human has ever come from the 67.202.xx.xx range.

A motley collection of bots have, though, both stealthy and self-proclaiming.