Welcome to WebmasterWorld Guest from 107.20.59.213

Forum Moderators: Ocean10000 & incrediBILL

Message Too Old, No Replies

Kindle & Amazon IPs?

     

keyplyr

12:10 pm on Nov 24, 2011 (gmt 0)

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month




So this is my 1st sighting of a Kindle user coming in with an Amazon IP non crawler, non AWS or at least the range isn't registered as AWS. User requested a popular web page and was stopped cold because the IP range is blocked; no other files requested. Could this be Fire? Or maybe Amazon is also an ISP in Ireland?


UA: Mozilla/5.0 (Linux; U; en-US) AppleWebKit/528.5+ (KHTML, like Gecko, Safari/528.5+) Version/4.0 Kindle/3.0 (screen 600x800; rotate)

Amazon (Ireland)
rDNS: no
87.238.80.0 - 87.238.81.255
87.238.80.0/21

wilderness

4:56 pm on Nov 24, 2011 (gmt 0)

WebmasterWorld Senior Member wilderness is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



keyplr,
There are some occasional compliant users @ Amazon.

I recall Jim keeping one range open for a bot that was beneficial to his sites, although I don't recall the bot or range.

incrediBILL

5:07 pm on Nov 24, 2011 (gmt 0)

WebmasterWorld Administrator incredibill is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Compliant Amazon users are kind of like Sasquatch sightings, people claim they see it but proof is elusive ;)

wilderness

5:47 pm on Nov 24, 2011 (gmt 0)

WebmasterWorld Senior Member wilderness is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Hey Bill,
I realize that "compliant" and "Amazon" are oxymoron's, however, and for the soothsayers, there's always exceptions.

incrediBILL

5:51 pm on Nov 24, 2011 (gmt 0)

WebmasterWorld Administrator incredibill is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Yes, I'm aware, but I'm not sure the exceptions that play in the Amazon cesspool deserve a break. If you reward that behavior they'll never learn, but it doesn't seem like the few trying to teach them are making any headway.

dstiles

9:59 pm on Nov 24, 2011 (gmt 0)

WebmasterWorld Senior Member dstiles is a WebmasterWorld Top Contributor of All Time 5+ Year Member



I'm with Bill on this one - Amazon-Anything stays blocked!

keyplyr

8:22 am on Nov 26, 2011 (gmt 0)

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



This intention of this thread was not so much about AWS, but Kindle's Fire.

The dynamic in question here is that Amazon's new Kindle Fire, which may become popular given the very low purchase price, purportedly requests our web pages via the Amazon cloud (EC2.)

It is yet-to-be-determined whether Amazon will try and cache (copy) our web pages or possibly just browse from EC2 proxy connections.

Pfui

9:47 am on Nov 26, 2011 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member



I'm not sure we've spotted Fire's UA yet -- unless they didn't change anything... [webmasterworld.com...]

Pfui

10:13 am on Nov 26, 2011 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member



Just found this specific info via the link below (bold emphasis mine):

Mozilla/5.0 (Linux; U; Android 2.3.4; en-us; Kindle Fire Build/GINGERBREAD) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1

Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_3; en-us; Silk/1.1.0-80) AppleWebKit/533.16 (KHTML, like Gecko) Version/5.0 Safari/533.16 Silk-Accelerated=true

Source: "The User Agent String of Kindle Fire Revealed" [11-18-11] [labnol.org...]

keyplyr

10:17 am on Nov 26, 2011 (gmt 0)

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Yes, same UA as previous discussion. The difference is where it came from, which presented the hypothesis.

::I feel so intelligent using words like that::

lucy24

10:52 am on Nov 26, 2011 (gmt 0)

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



I never made it past the IP range.

87.238.80.0 - 87.238.81.255
87.238.80.0/21


Shouldn't that be /23? Or possibly ..87.?

You know you've been locking out too many robots when you no longer have to spend hours with an abacus to figure these things out. :(

keyplyr

11:48 am on Nov 26, 2011 (gmt 0)

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Good eye lucy24 - Yes, 87.238.80.0 - 87.238.81.255 does resolve to 87.238.80.0/23

The 87.238.80.0/21 came from DNSstuff and probably reflects route. I was using their old browser add-on tool. Probably should do them all manually for better specificity.

[edited by: keyplyr at 11:53 am (utc) on Nov 26, 2011]

keyplyr

11:52 am on Nov 26, 2011 (gmt 0)

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Mozilla/5.0 (Linux; U; Android 2.3.4; en-us; Kindle Fire Build/GINGERBREAD) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1

Well that's simple enough.

Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_3; en-us; Silk/1.1.0-80) AppleWebKit/533.16 (KHTML, like Gecko) Version/5.0 Safari/533.16 Silk-Accelerated=true

Pfui

6:11 pm on Nov 26, 2011 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member



Some may need to tweak old UA blocks for: accelerate

(I did, re the now d/c'd Google Web Accelerator [webmasterworld.com...] and from way, way back to modem days.)

Pfui

7:34 pm on Dec 3, 2011 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member



This just in, emphasis mine. I still find it odd there's no indication of Kindle per se:

Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_3; en-us; Silk/1.1.0-84) AppleWebKit/533.16 (KHTML, like Gecko) Version/5.0 Safari/533.16 Silk-Accelerated=true

Really Big FYI: Silk hit html files, graphics, and favicon from amazonaws.com AND, seconds later, from Comcast:

ec2-50-16-61-105.compute-1.amazonaws.com
06:41:07 /dir/filename.html
06:41:14 /favicon.ico
06:41:35 /dir/filename.html
ec2-184-73-52-100.compute-1.amazonaws.com
06:46:56 /dir/filename.html
ec2-204-236-204-113.compute-1.amazonaws.com
07:26:49 /dir/filename.html

c-98-227-60-2nn.hsd1.il.comcast.net
06:41:11 /dir/filename.html
06:41:15 /favicon.ico
06:41:40 /dir/filename.html
06:47:01 /dir/filename.html
07:26:53 /dir/filename.html

AWS Silk = 403 because of AWS [webmasterworld.com...] independent of AWS-cached acceleration... Or non-acceleration: One of many such reports: [foxnews.com...]

Comcast Silk = 200. No clue what the latter actually 'saw' because ironically, the files viewed were highly formatted to mimic hard copy and would lose a LOT in any translation.

keyplyr

10:35 pm on Dec 3, 2011 (gmt 0)

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Too much of a coincidence to be 2 users. So it does appear Silk is, at least in some way, operating from the Amazon Cloud.

Scenario: Silk equipped Kindle user who's ISP is Comcast finds our site. The tablet gets all its internet capabilities from the Amazon Mother Ship which hits our site first and displays it on the tablet for the Comcast user.

Question: were the requested files the same? I assume they were from the same web page, but was there a division in files types between the Amazon IP and the Comcast IP?

Pfui

1:32 am on Dec 4, 2011 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member



The files requested were identical, even down to specific rewritten referrers (based on off-server requests for certain files, not the UAs doing the requesting).

For some reason I thought Kindle users only used/were limited to Amazon IPs from the get-go, so I was surprised to see the Comcast-based fellow traveler, let alone the totally resource-wasteful double-hits-double-hits. If they all tag-team, they're all getting rewritten, sorry.

Hmm. There appears to be the opposite setting out there: Silk-Accelerated=false

If/when I see that -- and it hits solo -- I may simply rewrite the =true types to a 'please switch gears' page. (But really, if you can enable acceleration, and They say it's faster, why wouldn't you? Who doesn't want to Go. Faster. Now!)

keyplyr

1:48 am on Dec 4, 2011 (gmt 0)

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



In a way, I like the fact that the user's ISP (in this case Comcast) is also requesting the same file. Since all Amazon IPs will be blocked at my server, hopefully the user still gets to see what they came for.

Pfui

1:46 am on Dec 7, 2011 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member



1.) Another Silk twin hit thing to report, even including identical G search referrers (that'll make stats crazy!). Every file hit first by AWS, then two to four seconds later by Frontier, using:

Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_3; en-us; Silk/1.1.0-84) AppleWebKit/533.16 (KHTML, like Gecko) Version/5.0 Safari/533.16 Silk-Accelerated=true

FIRST:

ec2-204-236-176-65.us-west-1.compute.amazonaws.com

SECONDS LATER:

50-103-7-1nn.dklb.il.frontiernet.net

FWIW, the identical G REF began:

http://www.google.com/url?sa=t&rct=j&q=[keyword=here]&source=web&cd=1&ved=[etc.]

(No UA apparent in REF; probably in lengthy alphanumeric string sections.)

2.) So okay, here's the thing. All non bot-related AWS hits are 403'd from the get-go, so technically, AWS shouldn't even 'know' there are graphics on pages it can't get to in the first place, right? But AWS does -- goes right for 'em:

AWS Silk:

1n:34:10 / 403
1n:34:17 /dir/exampleA.jpg 403
1n:34:17 /dir/exampleB.jpg 403
1n:34:17 /dir/exampleC.jpg 403
1n:34:17 /bots.jpg 200

FRONTIER Silk:

1n:34:12 / 200
1n:34:21 /dir/exampleA.jpg 200
1n:34:21 /dir/exampleB.jpg 200
1n:34:21 /dir/exampleC.jpg 200
[Didn't go for bots.jpg <= AWS hit was 200]

I've blocked amazonaws.com for two years. So there's NO way AWS could know file location details -- unless they're working off unauthorized caches/copies crawled by bare IPs or other Hosts. Dammit.

dstiles

9:39 pm on Dec 7, 2011 (gmt 0)

WebmasterWorld Senior Member dstiles is a WebmasterWorld Top Contributor of All Time 5+ Year Member



Only one "silk" hit so far this month (a double one, AWS first)...

--------------------
Date: 04/Dec/2011 17:37:58
IP: 107.22.104.199 (Amazon_AWS)
Proxy Fwd-For IP: 68.9.227.187::HTTP/1.1 silk

UA: Mozilla/5.0 (Linux; U; Android 2.3.4; en-us; Silk/1.1.0-84) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1 Silk-Accelerated=true

REFERER: [google.com...]

--------------------
Date: 04/Dec/2011 17:37:59
IP: 68.9.227.187 (Cox)

UA: Mozilla/5.0 (Linux; U; Android 2.3.4; en-us; Silk/1.1.0-84) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1 Silk-Accelerated=true

Referer: [google.com...]

--------------------

The fact that the referer was G's serps suggests there's an interaction going on.

Pfui

12:15 am on Dec 8, 2011 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member



The fact that the referer was G's serps suggests there's an interaction going on.


You mean like a real person searched G for something, and clicked through to you? (That's what I saw; more or less typical G-as-REF behavior, except it was doubled.)

dstiles

8:55 pm on Dec 8, 2011 (gmt 0)

WebmasterWorld Senior Member dstiles is a WebmasterWorld Top Contributor of All Time 5+ Year Member



To me it implies two services running in parallel - one from AWS and the other from the actual searcher (cox in my example).

This suggests something akin to a "toolbar" or "av-like" app running, one feeding back to AWS, the other doing what it's supposed to (ie, fetch a page). Timing may be disputable: if the ISP is slower than AWS then AWS could get its request in first (my example); if the AWS route was slower then vice versa.

So what is really happening? Is an AWS app, true to Cloud form, scraping data, perhaps for "popularity" testing? What relationship does it have to these browsers? What, in fact ARE the browsers? Is Silk a plug-in app or part of the browser? Is the world coming to an end? Who knows?! :(

Pfui

10:04 pm on Dec 8, 2011 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member



FWIW... [webmasterworld.com...]

dstiles

10:50 pm on Dec 8, 2011 (gmt 0)

WebmasterWorld Senior Member dstiles is a WebmasterWorld Top Contributor of All Time 5+ Year Member



Thanks. I'd almost forgotten that - memory is not one of my good points. :)

Still doesn't explain the parallel-ishness. :(

If silk (as whatever browser it pretends to be) is running from within the cloud one would expect ONLY the cloud IP. Since it's coming at us with both cloud AND ISP IPs there has to be more to it, and my surmise is it's not going to be to webmasters' advantage.

A thought occurs: we all block AWS here, right? So: if AWS is merely acting as a pre-loader, when it reports to the browser that it can't get the page, perhaps that's when the browser tries to access it. Is there anyone here who can test that? If AWS is not blocked, does the broadband IP still come in?

Pfui

6:17 pm on Dec 21, 2011 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member



FWIW: I finally saw one of these, from an AT&T address by itself -- no Silk, no twin hits from AWS:

76.249.189.2nn
Mozilla/5.0 (Linux; U; Android 2.3.4; en-us; Kindle Fire Build/GINGERBREAD) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1

keyplyr

10:00 pm on Dec 21, 2011 (gmt 0)

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



Should increase quite a lot after Christmas.

Pfui

8:00 pm on Jan 5, 2012 (gmt 0)

WebmasterWorld Senior Member 5+ Year Member



I've seen zilch increase. Am surprised.

Btw..

"Amazon Kindle Fire browser hacked for your Android pleasure" [theregister.co.uk...]

dstiles

8:45 pm on Jan 5, 2012 (gmt 0)

WebmasterWorld Senior Member dstiles is a WebmasterWorld Top Contributor of All Time 5+ Year Member



"Silk offloads most of the rendering of web pages onto the cloud..."

Which explains some things we've discussed and adds another way of killing potential visitors to our web sites. :(
 

Featured Threads

Hot Threads This Week

Hot Threads This Month