another day, another botnet

[lucy24]

Like the man said, the robots ye will always have with you.

This vexing botnet started showing up near the end of December. The whole package shows up anywhere from one to seven times a day, very rarely more--which is, of course, already seven times too often.

Typical example in logs:

aa.bb.cc.dd - - [17/Mar/2020:11:02:20 -0700] "GET /fun/robots/ HTTP/1.1" 403 6706 "https://example.com/" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3381.1 Safari/537.36" 
aa.bb.cc.dd - - [17/Mar/2020:11:02:21 -0700] "GET / HTTP/1.1" 403 6706 "https://example.com" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3381.1 Safari/537.36" 
aa.bb.cc.dd - - [17/Mar/2020:11:07:56 -0700] "GET / HTTP/1.1" 200 10382 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/65.0.3325.181 Safari/537.36" 

Pattern:
GET /fun/robots/ with / as referer
GET / with root (minus final slash) as referer
and then, after anywhere from a few minutes to--rarely--over an hour:
GET / without referer
The first two are consistently blocked on referer grounds. The third is the one that catches my notice.

All requests come straight to https. (The site changed over at the end of October.)

IP:
varies from one visit to the next, though they are sometimes reused on subsequent days. A fair number of the ones I’ve looked up are ISPs in assorted poor countries (the ones that, for various reasons, are more vulnerable to hacking).

UA:
I haven’t paid any particular attention to the ones used for the first pair of requests--the ones that would be blocked anyway. The third request has a plausible humanoid UA that is used repeatedly for a day or two.

Head-scratcher:
This is obviously an established pattern, and I do not for one moment believe that my site has been singled out. But the initial URL is always the same page, and it’s one that is specific to this site. (That is: It may not be the only /fun/robots/ in the world, but it is certainly less predictable than /wp-admin/.) I would be interested to know if others have seen an equivalent pattern.

[dstiles]

Is it always chrome 65 for mac and 67 for the others? In my book, obsolete browsers.

[lucy24]

Is it always chrome 65 for mac and 67 for the others?

No. Like I said, the UA in the third request changes every day or two. It isn't always Mac; sometimes they use Windows or Linux. I've even seen Vivaldi, and I think there were some Operas.

I used this pattern in logs:

^([\d.]+) - - \S+ \S+ "GET /fun/robots/ HTTP/1.1" 403 \d+ "https://example.com/".+\n\1 - - \S+ \S+ "GET / HTTP/1.1" 403 \d+ "https://example.com".+\n(?:.+\n)+?\1 - - \S+ \S+ "GET / HTTP/1.1" 200 

I'm not sure about the best way to test it on other sites; /fun/robots/ would obviously have to be replaced with something more general, but still limited to pages, like
/\w+/([\w-]/)*([\w-]+\.html)? HTTP/
and then once you've got a hit, start looking for that specific page, whatever it turns out to be.

In any case, Chrome 65 for Mac is not “obsolete”. In fact, it is the most recent version that users with my OS (10.9) are allowed to download. I have 67, but if I downloaded today, 65 is the last I would be allowed to get.

[dstiles]

I don't envy you looking for that! :)

According to whatismybrowser...
Stable branch of Chrome:
Platform Version Release Date
Chrome on Windows 80.0.3987.137 2020-03-14
Chrome on macOS 80.0.3987.137 2020-03-14
Chrome on Linux 80.0.3987.137 2020-03-14
Chrome on Android 80.0.3987.132 2020-03-04
Chrome on iOS 80.0.3987.95 2020-02-11

I currently block up to 69, which I consider more than a safe margin. Earlier Chrome browsers (and even the latest) have horrendous bugs. I had it installed (on linux) for a few months and used it perhaps a dozen times before removing it again.

Vivaldi, by the way, has rebranded its UA to look exactly like chrome - too may web sites were rejecting it. :(

[lucy24]

Earlier Chrome browsers (and even the latest) have horrendous bugs.

All the better for botnets to hack into your computer :)