Welcome to WebmasterWorld Guest from 54.221.49.52

Forum Moderators: Ocean10000 & incrediBILL

Amazon AWS Hosts Bad Bots

Continuation Thread

     

incrediBILL

11:16 pm on May 16, 2013 (gmt 0)

WebmasterWorld Administrator incredibill is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



This is a continuation from the previous thread:
[webmasterworld.com...]

Post about spiders coming from Amazon's AWS hosting.

dstiles

9:57 pm on Jan 7, 2014 (gmt 0)

WebmasterWorld Senior Member dstiles is a WebmasterWorld Top Contributor of All Time 5+ Year Member



keyplr - it's cloud. The range could easily be some technical service or proxy service. If it's genuine mobile access then someone is being a bit naive.

And which thetrasher's link confirms. Nokia are obviously trying to do things on the cheap. If they provide a proper IP range they will get through.

It all comes down to trust. It's amazon: I don't.

keyplyr

2:32 am on Jan 8, 2014 (gmt 0)

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month





The mobile hits I see do contain "proxy" in the UA string.

Angonasec

12:57 am on Jan 9, 2014 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Sample using a new Nokia, and you'll breathe-easy discovering why any "human" using one will soon deposit the Nokia in the dustbin.

As intuitive and effortlessly interactive as bats in the pantry.

keyplyr

10:01 am on Jan 9, 2014 (gmt 0)

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month





@dstiles
Well I poked a hole & let a few through. Look human to me, no issues yet. Keeping a close watch.

dstiles

3:46 pm on Jan 30, 2014 (gmt 0)

WebmasterWorld Senior Member dstiles is a WebmasterWorld Top Contributor of All Time 5+ Year Member



Three hits today from a major new amazon range, registered in November.

54.72.0.0 - 54.95.255.255

keyplyr

4:59 pm on Jan 30, 2014 (gmt 0)

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Thanks dstiles, new for me.

lucy24

8:06 pm on Jan 30, 2014 (gmt 0)

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



I had to look that up because it seemed so odd. Looks like the bottom half of 72-79 is still Merck-- at least this week-- but honestly, would anything bad happen if you just lock out the whole 54.0.0.0/8 and be done with it? I've never personally met anyone from <192.

:: idly wondering how Merck stock is doing these days ::

dstiles

8:12 pm on Jan 30, 2014 (gmt 0)

WebmasterWorld Senior Member dstiles is a WebmasterWorld Top Contributor of All Time 5+ Year Member



The ranges either side of the amazon one are merck but I've left them alone, since they leave me alone. I have only two merck ranges listed and both are enabled.

dstiles

2:56 pm on Feb 21, 2014 (gmt 0)

WebmasterWorld Senior Member dstiles is a WebmasterWorld Top Contributor of All Time 5+ Year Member



Another large amazon range...

NetRange: 54.176.0.0 - 54.191.255.255
CIDR: 54.176.0.0/12
OriginAS: AS16509
NetName: AMAZON-2011L
RegDate: 2013-11-25

That runs straight into one I logged Dec 2012...

54.192.0.0 - 54.255.255.255

keyplyr

6:38 pm on Feb 21, 2014 (gmt 0)

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Cool, that covers a lot

webcentric

5:45 pm on Apr 5, 2014 (gmt 0)

WebmasterWorld Senior Member Top Contributors Of The Month



In looking at ranges inside 54.192.0.0/10, I notice a gap which appears to have been transferred to APNIC.

54.222.0.0 - 54.223.255.255 -- 54.222.0.0/16
netname: CHINANETCENTER
descr: Wangsu Science & Technology Co.,Ltd.

Thought it was worth mentioning.

dstiles

7:39 pm on Apr 5, 2014 (gmt 0)

WebmasterWorld Senior Member dstiles is a WebmasterWorld Top Contributor of All Time 5+ Year Member



It has the same response: block it.

Incidentally, it's /15 not /16 :)

webcentric

8:22 pm on Apr 5, 2014 (gmt 0)

WebmasterWorld Senior Member Top Contributors Of The Month



Oops, ran the query for /16 and forgot it returned /15. 54.192.0.0/10 works for me. ;) Thanks.

keyplyr

9:47 pm on Apr 5, 2014 (gmt 0)

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month




@webcentric - there's also Nokia Express mobile ISP in there:

54.209.248.0/22
54.209.248.0 - 54.209.251.255

54.236.252.0/22
54.236.252.0 - 54.236.255.255

54.244.56.0/21
54.244.56.0 - 54.244.63.255

However, that being said... I also block the entire /10

lucy24

8:59 pm on Apr 27, 2014 (gmt 0)

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



Another large amazon range...

NetRange: 54.176.0.0 - 54.191.255.255

I'd somehow overlooked this until I met one today :( Re-check in free lookup says that 96-175 is still Merck, leading to the question:

Has anyone, ever, met a legitimate human from anywhere in the 54 block? I don't mean in 1992 when this range was first allocated; I mean recently. Maybe if you've got a reputable medical-information site-- which I don't.

:: irresistible detour tells me that-- surprise! --Merck stock is performing respectably and even pays dividends ::

kazzo

2:22 pm on Jul 3, 2014 (gmt 0)



New poster here. Found the site while trying to figure out an issue we had recently. Our company website got bombarded on the 1st of July for roughly 25 minutes - all IPs from ranges belonging to amazonaws. It looked like whatever or whoever it was scraped our site for all our product images. Burned a lot of our bandwidth too.

Ranges:

23.20.131.219 - 23.22.145.240

54.83.74.76 - 54.242.89.161

and one sole source at 107.20.19.114

keyplyr

2:51 pm on Jul 3, 2014 (gmt 0)

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Hi kazzo, welcome to WebmasterWorld.

These Amazon ranges have been listed earlier in this thread. You can use the site search utility at the top to find them, possibly searching for the A or A & B subnets.

23.20.131.219 - 23.22.145.240 is part of a greater Amazon range:
23.20.0.0 - 23.23.255.255
23.20.0.0/14

54.83.74.76 - 54.242.89.161 is part of a greater Amazon range:
54.72.0.0 - 54.95.255.255
54.80.0.0/12

107.20.19.114 is part of a greater Amazon range:
107.20.0.0 - 107.23.255.255
107.20.0.0/14

Angonasec

4:32 am on Jul 4, 2014 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



# AISearchBot AmznAWS
deny from 23.20.0.0/14 50.16.0.0/14 50.112.0.0/16 54. 65.19.128.0/18 67.202.0.0/18 72.44.32.0/19 75.101.128.0/17 107.20.0.0/14 174.129. 184.72.0.0/15 184.169.128.0/17 204.236.128.0/17

Blocking this fat tranche does NOT affect our Amazon affiliate status.

keyplyr

5:37 am on Jul 4, 2014 (gmt 0)

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month




deny from 23.20.0.0/14 50.16.0.0/14 50.112.0.0/16 54. 65.19.128.0/18 67.202.0.0/18 72.44.32.0/19 75.101.128.0/17 107.20.0.0/14 174.129. 184.72.0.0/15 184.169.128.0/17 204.236.128.0/17

?

Angonasec

11:22 am on Jul 4, 2014 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



54. Inhuman as noted in this thread et al.
174.129.0.0/16 is all Amazon noise.

So why the ?

keyplyr

7:07 pm on Jul 4, 2014 (gmt 0)

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Wouldn't want to mislead others into thinking 54 is exclusively AWS, it isn't. Just a quick example...

Nokia Express mobile carrier:
54.209.248.0 - 54.209.251.255
54.209.248.0/22
54.236.252.0 - 54.236.255.255
54.236.252.0/22
54.244.56.0 - 54.244.63.255
54.244.56.0/21

There are others. Don't know about you, but I appreciate a large mobile customer base.

dstiles

8:02 pm on Jul 4, 2014 (gmt 0)

WebmasterWorld Senior Member dstiles is a WebmasterWorld Top Contributor of All Time 5+ Year Member



Those ranges are still amazon-owned, though, and at least the first one says "services" which is not necessarily the same as "mobile broadband". It could be those ranges are actually non-public - eg nokia in-house or a VPN network.

Just an observation. Either way, they are blocked here.

keyplyr

9:25 pm on Jul 4, 2014 (gmt 0)

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



@ dstiles - Personally, I get upwards of 60% mobile traffic on one site and over 40% of sales overall from mobile device users. Obviously Nokia popularity is geo specific so your particular user base may not be affected by blocking them, but this is a public forum often used as a knowledge base so IMO the correction to "54. Inhuman" was warranted.

Just a FYI - I block most all of 54, I just poke a few holes. Additionally, there are several other "holes" in 54. & 174.129. depending on your users.

not2easy

11:07 pm on Jul 5, 2014 (gmt 0)

WebmasterWorld Administrator 5+ Year Member Top Contributors Of The Month



Just a note re:
54.72.0.0 - 54.95.255.255
54.80.0.0/12

It needs 54.72.0.0/13 also to cover all of it, had to look it up today.

keyplyr

12:34 am on Jul 6, 2014 (gmt 0)

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Thanks not2easy

I was just pointing to where the earlier sub-range belonged.

Pfui

11:05 pm on Aug 15, 2014 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member



ec2-54-164-73-8.compute-1.amazonaws.com (a.k.a 54.164.73.8)
Manticore 0.3.1

robots.txt? NO

(Will also post as a standalone thread for UA-related comments.)

keyplyr

1:20 am on Aug 16, 2014 (gmt 0)

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month





Thanks Pfui, didn't have this one:

54.160.0.0/12
54.160.0.0 - 54.175.255.255

***

So basically we have this:
54.160.0.0/12
54.160.0.0 - 54.175.255.255
54.176.0.0/12
4.176.0.0 - 54.191.255.255
54.192.0.0/10
54.192.0.0 - 54.255.255.255


Which can be efficiently minified to:
54.160.0.0 - 54.255.255.255
54.160.0.0/11
54.192.0.0/10

Pfui

11:09 pm on Aug 19, 2014 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member



FWIW: 54.167 has been busy of late, (ditto 54.166), including the following which doesn't really need a separate thread. Changing mixed-case to all-lower, it's clueless in its actions, and naming:

User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31

(See also imminent thread: HubSpot Webcrawler)

not2easy

1:27 am on Aug 20, 2014 (gmt 0)

WebmasterWorld Administrator 5+ Year Member Top Contributors Of The Month



I saw a string of visits today from their 54.164. range, strange behavior of one visit, go home, change shoes and come right back with slightly different IP numbers for 1 more .html, rinse, repeat.
UA: "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/21.0.1180.75 Safari/537.1"

dstiles

4:08 pm on Nov 19, 2014 (gmt 0)

WebmasterWorld Senior Member dstiles is a WebmasterWorld Top Contributor of All Time 5+ Year Member



New amazon IP range registered in October:

NetRange: 54.144.0.0 - 54.159.255.255
CIDR: 54.144.0.0/12
Organization: Amazon Technologies Inc. (AT-88-Z)
OrgTechName: Amazon EC2 Network Operations

The latter suggests cloud.
This 108 message thread spans 4 pages: 108
 

Featured Threads

Hot Threads This Week

Hot Threads This Month