AdlesseBot

[keyplyr]

Been seeing this one for about a week; blocked of course :)

UA: Mozilla/5.0 (compatible; AdlesseBot/1.0; [e-mail]) v1441405262.1173
Host: AWS
54.160.0.0 - 54.175.255.255
54.160.0.0/12
Robots.txt: No

Seems to be a service that offers both desktop & mobile browser add-ons to block Ads. However, unlike Ad-Aware, Adblock Plus and others, this one wants to download all webpages & other related files. The purpose for this is unclear, but one possibility is caching, then stripping out ads, then serving the web page to the end user. Another possibility would be for collecting ad data for product R&D.

[keyplyr]

BTW - the "v" number seems to change randomly each request, obviously not version number :)

[keyplyr]

I believe this agent is also spoofing UAs, coming in stealth. So if you have ads or affiliate links, beware.

[lucy24]

the "v" number seems to change randomly each request, obviously not version number

I seem to remember one Polish robot pulling something similar: they'd take the exact request time and make it part of the UA string, apparently in the foolish hope that this would prevent UA-based blocking. (Also seem to remember that this detail revealed that the robot's clock was off by several minutes :))

[keyplyr]

I used to pull info like that using perl, then I forgot how and started doing it using php. Last week I had to re-learn (again) how to pass another type of array for page URI written into a JS that loads for every page, but using unique page names so I stole a bit of code from the Facebook share api. It's all the same thing really, just a dozen different ways of doing it.