Mail.RU

new range

[lucy24]

As of mid-23 May (US time, may correspond to the exact rollover to 24 May in their time zone):

UA (unchanged): Mozilla/5.0 (compatible; Linux x86_64; Mail.RU_Bot/2.0; +http://go.mail.ru/help/robots)
IP: 95.163.248-255
95.163.248.0/21

The above is their official range. The de facto crawl range is much smaller; I see only
95.163.255.72-79
95.163.255.72/29
if anyone wants to be that finical.

Previous IP: 217.69.133 (again, the de facto range is smaller than their official range, which I think was a /20).

Incidentally, this robot has an annoying though not unique habit which I hadn't noticed before: when it gets redirected, the previous request is plugged into the Referer slot. (The most irritating aspect of this behavior is that when you call someone on it, they insist--wrongly--that that's what you are supposed to do.)

[keyplyr]

The full range is:
95.163.64.0 - 95.163.255.255
95.163.128.0/17, 95.163.64.0/18

Problem is, there are human visitors coming from msm.ru (formerly di-net.ru) a huge Russian/Ukrainian ISP that has a half dozen scattered C & Ds in this range (maybe you don't get these guys, but I get a few a month.)

I've been allowing the full range with prejudice.

- - -

My other Mail.ru ranges

195.211.20.0 - 195.211.23.255
195.211.20.0/22

217.69.128.0 - 217.69.143.255
217.69.128.0/20

[lucy24]

a half dozen scattered C & Ds

Half a dozen scattered whats?

[keyplyr]

C sub ranges and D sub ranges

as in... A.B.C.D

[lucy24]

Oh, oops, I thought it meant something unkind, like Cease & Desist :)

Before meeting the Mail.RU_Bot from this range, I'd only got the /16 labeled as Russia; never had occasion to look in more detail.

[keyplyr]

I thought it meant something unkind...

Well we were talking about ranges so I didn't see the need to be explicit.

I've had the mail.ru UA attribute blocked for sometime.

But now that you've brought it up, I don't think I've seen anything more than the occassional WordPress or PHP vulnerability probe from any of those ranges, which makes them no worse than any other range, even though they're owned by mail.ru.

Have you seen any other bots or
ne'er-do-wellers from there?


[fix typo]

[edited by: keyplyr at 8:11 am (utc) on May 29, 2018]

[lucy24]

Have you seen any other bots or ne'er-do-wellers from there?

:: detour to recent logs ::

Nothing in the past year-plus. Well, one cluster of attempts from 95.163.216.abc last August, but 403 is out of sight, out of mind.

Some years back I had Mail.RU blocked (by IP) due to irritating behavior w/r/t images--can't remember anything more detailed--but whatever it was, they've long since stopped doing it.

[keyplyr]

95.163.255.64 - - [31/May/2018:20:29:51 -0700] "GET /robots.txt HTTP/1.0" 200 8767 "-" "Mozilla/5.0 (compatible; Linux x86_64; Mail.RU_Bot/2.0; +http://go.mail.ru/help/robots)"
95.163.255.66 - - [31/May/2018:20:30:02 -0700] "GET /example.html HTTP/1.1" 403 4726 "-" "Mozilla/5.0 (compatible; Linux x86_64; Mail.RU_Bot/2.0; +http://go.mail.ru/help/robots)"

I fairly sure at one time I had Mail.RU_Bot disallowed in robots.txt, but no longer... probably because it was ignored. Just disallowed it again to see if it has changed its ways.

Always amused me how their help/robots page explains what robots are but very blatantly avoids detailing exactly what *their* robot wants our content for.

[lucy24]

very blatantly avoids detailing exactly what *their* robot wants our content for.

I always assumed it was a minor search engine, possibly one of those that are attached to an ISP.