AdvBot

[lucy24]

Anyone know anything? Forums search turned up absolutely nothing ... except one sample Deny list that included the "advbot" UA.

136.243.14.164 - - [21/Jan/2015:05:41:24 -0800] "GET / HTTP/1.1" 301 508 "-" "Mozilla/5.0 (compatible; AdvBot/2.0; +http://advbot.net/bot.html)" 
136.243.14.164 - - [21/Jan/2015:05:41:25 -0800] "GET /robots.txt HTTP/1.1" 200 582 "-" "Mozilla/5.0 (compatible; AdvBot/2.0; +http://advbot.net/bot.html)" 
136.243.14.164 - - [21/Jan/2015:05:41:30 -0800] "GET / HTTP/1.1" 200 2488 "-" "Mozilla/5.0 (compatible; AdvBot/2.0; +http://advbot.net/bot.html)" 

... et cetera for a total of about 50 requests in sundry directories, following no discernible pattern.

Aside from the initial not-quite-kosherness of requesting the front page before looking at robots.txt, their behavior was unimpeachable. I've nevertheless blocked them because (a) they live at Hetzner (it's an old range that I neglected to block when I first learned of it) and (b) the URL given in the UA leads only to an authentication request followed by generic 401 page.

Free lookup gives a series of contact emails at yandex.ru, but I am inclined to believe this is either bogus or unauthorized.

[keyplyr]

For Eastern Europe, a yandex.ru email address is like a gmail.com here in the US IMO. I see it used fairly often as company & even server admin contact info.

Seems dodgy to require credentials on a bot info page, so even if I didn't already block that server farm range, I'd probably block because of that.

The name advbot suggests advertising.

[lucy24]

a yandex.ru email address is like a gmail.com here in the US

Oh, thanks, that wouldn't have occurred to me.

The name advbot suggests advertising.

It sure does, doesn't it?

[trintragula]

They showed up at my site for the first time from a different Hetzner address - 5.9.7.nnn - after you started this thread.
They started on my home page and politely explored the small number of links that are visible there when you're blocked.
Well-behaved and non-redundant.

[keyplyr]

Well-behaved? Probably attempting to scrape your content for their advertisers.

[trintragula]

Very likely. Polite though. ;)

[wilderness]

robots and root, with the later 403'd.

5.9.24.198 - - [24/Jan/2015:05:33:51 -0700] "GET / HTTP/1.1" 200 590 "-" "Mozilla/5.0 (compatible; AdvBot/2.0; +http://advbot.net/bot.html)"

[keyplyr]

So it appears to be distro

[trintragula]

All Hetzner so far, though - unless anyone's seen it elsewhere.