Amazonbot

[Peter_S]

Hi,

Lot of visits (10k+) from Amazonbot, during the last 24 hours.

Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1)
AppleWebKit/600.2.5 (KHTML, like Gecko) Version/8.0.2 Safari/600.2.5
(Amazonbot/0.1; +https://developer.amazon.com/support/amazonbot)

Amazonbot is Amazon's web crawler used to improve our services, such as enabling Alexa to answer even more questions for customers.

Does it profit publishers in one way or another? It doesn't look like it to me.

Not to be confused with Amazon AdBot : [webmasterworld.com...]

[SumGuy]

I haven't looked specifically for amazon as a referrer. I block all Amazon IP's from hitting my web server, except those that might be useful (ie if something like DuckDuckGo operated from an Amazon IP).

But upon reading this post I was curious about Apple / Siri. I've never seen apple or applebot as a referrer either. I had a friend with an iphone ask siri a question that I know should result in a pointer to my website. When he got siri's suggestions, with one of them a link to my website, he selected it, and I guess it was brought up on his phone using Safari browser. I looked at the logs and there was nothing in the referrer field for that hit.

[lucy24]

:: quick run to raw logs ::

Oh, oops. I started testing Amazonbot for robots.txt compliance last summer and then forgot all about it. In the present calendar year they have requested nothing but robots.txt, so I guess I should authorize them and see where they go.

[lucy24]

Unexpected follow-up:

In the most recent logs (this past Sunday afternoon), I found a full visit from the newly authorized Amazonbot--not on my main site but my personal site, which only has a couple dozen visible pages. They used three different IPs in the Usual Suspects ranges, and went only where allowed. Requests came in two clumps; within each clump, spaced a second or two apart.