homepage Welcome to WebmasterWorld Guest from 54.83.133.189
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor
Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL

Search Engine Spider and User Agent Identification Forum

This 44 message thread spans 2 pages: < < 44 ( 1 [2]     
Amazon AWS Hosts Bad Bots
Continuation Thread
incrediBILL




msg:4574829
 11:16 pm on May 16, 2013 (gmt 0)

This is a continuation from the previous thread:
[webmasterworld.com...]

Post about spiders coming from Amazon's AWS hosting.

 

dstiles




msg:4635830
 9:57 pm on Jan 7, 2014 (gmt 0)

keyplr - it's cloud. The range could easily be some technical service or proxy service. If it's genuine mobile access then someone is being a bit naive.

And which thetrasher's link confirms. Nokia are obviously trying to do things on the cheap. If they provide a proper IP range they will get through.

It all comes down to trust. It's amazon: I don't.

keyplyr




msg:4635868
 2:32 am on Jan 8, 2014 (gmt 0)



The mobile hits I see do contain "proxy" in the UA string.

Angonasec




msg:4636058
 12:57 am on Jan 9, 2014 (gmt 0)

Sample using a new Nokia, and you'll breathe-easy discovering why any "human" using one will soon deposit the Nokia in the dustbin.

As intuitive and effortlessly interactive as bats in the pantry.

keyplyr




msg:4636137
 10:01 am on Jan 9, 2014 (gmt 0)



@dstiles
Well I poked a hole & let a few through. Look human to me, no issues yet. Keeping a close watch.

dstiles




msg:4641250
 3:46 pm on Jan 30, 2014 (gmt 0)

Three hits today from a major new amazon range, registered in November.

54.72.0.0 - 54.95.255.255

keyplyr




msg:4641278
 4:59 pm on Jan 30, 2014 (gmt 0)

Thanks dstiles, new for me.

lucy24




msg:4641308
 8:06 pm on Jan 30, 2014 (gmt 0)

I had to look that up because it seemed so odd. Looks like the bottom half of 72-79 is still Merck-- at least this week-- but honestly, would anything bad happen if you just lock out the whole 54.0.0.0/8 and be done with it? I've never personally met anyone from <192.

:: idly wondering how Merck stock is doing these days ::

dstiles




msg:4641312
 8:12 pm on Jan 30, 2014 (gmt 0)

The ranges either side of the amazon one are merck but I've left them alone, since they leave me alone. I have only two merck ranges listed and both are enabled.

dstiles




msg:4647482
 2:56 pm on Feb 21, 2014 (gmt 0)

Another large amazon range...

NetRange: 54.176.0.0 - 54.191.255.255
CIDR: 54.176.0.0/12
OriginAS: AS16509
NetName: AMAZON-2011L
RegDate: 2013-11-25

That runs straight into one I logged Dec 2012...

54.192.0.0 - 54.255.255.255

keyplyr




msg:4647566
 6:38 pm on Feb 21, 2014 (gmt 0)

Cool, that covers a lot

webcentric




msg:4660734
 5:45 pm on Apr 5, 2014 (gmt 0)

In looking at ranges inside 54.192.0.0/10, I notice a gap which appears to have been transferred to APNIC.

54.222.0.0 - 54.223.255.255 -- 54.222.0.0/16
netname: CHINANETCENTER
descr: Wangsu Science & Technology Co.,Ltd.

Thought it was worth mentioning.

dstiles




msg:4660755
 7:39 pm on Apr 5, 2014 (gmt 0)

It has the same response: block it.

Incidentally, it's /15 not /16 :)

webcentric




msg:4660758
 8:22 pm on Apr 5, 2014 (gmt 0)

Oops, ran the query for /16 and forgot it returned /15. 54.192.0.0/10 works for me. ;) Thanks.

keyplyr




msg:4660764
 9:47 pm on Apr 5, 2014 (gmt 0)


@webcentric - there's also Nokia Express mobile ISP in there:

54.209.248.0/22
54.209.248.0 - 54.209.251.255

54.236.252.0/22
54.236.252.0 - 54.236.255.255

54.244.56.0/21
54.244.56.0 - 54.244.63.255

However, that being said... I also block the entire /10

This 44 message thread spans 2 pages: < < 44 ( 1 [2]
Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Search Engine Spider and User Agent Identification
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
© Webmaster World 1996-2014 all rights reserved