Interesting attack pattern

xmlrpc.php attack

[jackson]

I've just set up my first VPS based website. Needless to say, it's been fun.

Over the past couple of days I've been 'hit' by an xmlrpc.php brute force attach. It was sufficient to take down my fail2ban app - which, I may add, still needs to be set up properly.

At first, I managed to "successfully" do a redirect by "reconfiguring" the xmlrpc.php file. While this kind of saved my resources - CPU, memory and swamping the MySQL database - the set of IP addresses were still hammering away at my set up.

With this persistence, I decided to do a simple 'Deny IP address" line in the .htaccess file. As soon as this took affect, something interesting happened. The attach persisted but, with an interesting switch. Not sure what to make of this? How, what, where and why is protected2.by0x6ab.com?

I did a Google on some of my findings but, came up empty handed. Just wondering if someone here can throw some light on this matter? Here's a snip of that transition:

185.244.43.71 - - [30/Nov/2018:22:31:25 +0800] "POST /xmlrpc.php HTTP/1.0" 302 197 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"
185.244.43.71 - - [30/Nov/2018:22:31:25 +0800] "POST /xmlrpc.php HTTP/1.0" 302 197 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"
185.244.43.71 - - [30/Nov/2018:22:31:26 +0800] "POST /xmlrpc.php HTTP/1.0" 302 197 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"
185.244.43.71 - - [30/Nov/2018:22:31:26 +0800] "POST /xmlrpc.php HTTP/1.0" 302 197 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"
protected2.by0x6ab.com - - [30/Nov/2018:22:31:27 +0800] "POST /xmlrpc.php HTTP/1.0" 403 480 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"
protected2.by0x6ab.com - - [30/Nov/2018:22:31:27 +0800] "POST /xmlrpc.php HTTP/1.0" 403 478 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"
protected2.by0x6ab.com - - [30/Nov/2018:22:31:29 +0800] "POST /xmlrpc.php HTTP/1.0" 302 197 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"
protected2.by0x6ab.com - - [30/Nov/2018:22:31:29 +0800] "POST /xmlrpc.php HTTP/1.0" 302 197 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"
protected.by0x6ab.xyz - - [30/Nov/2018:22:31:31 +0800] "POST /xmlrpc.php HTTP/1.0" 302 197 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"
protected.by0x6ab.xyz - - [30/Nov/2018:22:31:31 +0800] "POST /xmlrpc.php HTTP/1.0" 302 197 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"
protected.by0x6ab.xyz - - [30/Nov/2018:22:31:31 +0800] "POST /xmlrpc.php HTTP/1.0" 302 197 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"
protected.by0x6ab.xyz - - [30/Nov/2018:22:31:31 +0800] "POST /xmlrpc.php HTTP/1.0" 302 197 "-" "Mozilla/4.0 (compatible: MSIE 7.0; Windows NT 6.0)"

This particular attack came from a trio of IP addresses all starting with 185.244.43.x. The earlier attack came from the same ISP but spanning 8 different IP addresses.

Comments and suggestions would be appreciated.

Cheers

[not2easy]

The pattern you describe is fairly typical, only the 302's shown are unusual and likely due to the method used for blocking. Since they all use the HTTP1.0 protocol it would be easier to block all of those.

You don't share your actual lines for deny, but generally speaking it is far more efficient to determine the full range of IPs included for unwanted IPs and block the entire CIDR. Rather than a line for each and no deny at all for other IPs available to them, you are shooting at elephants with BBs.

The IPs shown are from UnionGroup RU:
185.244.40.0 - 185.244.43.255
RU-UNIONGR-20180131
185.244.40.0/22

The CIDR to block them all would be 185.244.40.0/22 so a line to "deny from 185.244.40.0/22" would give them all a 403, BUT that does not make them stop asking.

The "protected2.by0x6ab.com" looks like a typical whois privacy copany that keeps the personal registry information private at registrars.

If you do not use your WP xmlrpc.php for anything, it may be a good idea to rename it to something else, say "rmpxl.php" and give them all a 404 instead, until you examine your logs for the IPs getting those 404s.

If you're on WP, you need to anticipate that the default files will be targets of every hacker there is and you may want to get into the habit of examining your access logs to identify and learn to properly block either by unwanted IP or UA.

There are probably thousands of threads here that discuss blocking methods, it might pay to read through some of them to get a good grasp of the what, why and how. ;)

[lucy24]

If you do not use your WP xmlrpc.php for anything

Ah, is that what it's for? I see (blocked) requests for it all the time.

:: detour to recent archived logs ::

I thought I was being hyperbolic, but yup, every day. Generally GET, sometimes POST. Often with the query ?rsd --that is, not “some-parameter=rsd”, just rsd and-that's-all. All, without exception, blocked. (I did a supplementary search for (?!403) and got a few hits--but all were 418, which trumps 403 because my host uses it for mod_security.)

But it's not just whether you personally use it; it's whether it is directly used by visitors. If it's only used internally, then nobody should be allowed to get (i.e. GET) it at all. The generic Apache pattern is

RewriteCond %{THE_REQUEST} filename.xtn
RewriteRule filename\.xtn - [F,NS]

which basically means “If you ask for it by name, you can’t have it”. (Depending on circumstances, the NS flag may be superfluous, but it can’t hurt.)

[not2easy]

The xmlrpc.php is used to allow WP admin (and any users that have permissions set) to write posts remotely, without logging in to WP which is why it is such a target. It is only used remotely.

It is an option which imo is only used by WP users who don't know anything about security or else users who are very savvy about security. It is a default WP file in every installation, just like wp-login.php which is why they are so highly targeted. It is useless if not set up, but once set up, becomes vulnerable to bot attacks because people set silly passwords. I would never use it, that is too lazy even for me.

[lucy24]

Eeuw. That sounds like just the kind of thing that, if you did choose to use it, you would first rename and relocate the file so people can't just walk in off the street* and ask for it. (If I had it to do over again I would give my /includes directory a different name, but as it is, I intercept all external requests with a manual 404.) It probably also helps if you only give access to people with relatively fixed IP addresses--no posting from your mobile device unless you happen to be at home.

* Since we are talking about malign robots, “crawl up out of the gutter” may be a better expression.

[lucy24]

Returning to the original question:

Here's a snip of that transition:

The two different openings have less to do with the attacking robot than with your logging format. The first few lines show the raw IP address; the later lines show what the IP address resolves to. Assuming you didn't nip in and change something in your config file at precisely 30/Nov/2018:22:31:26 your-local-time, it means the attackers moved to a different IP address--one that, unlike the earlier one, resolves to a hostname of some kind.

The question is, do you want your logs to be in resolve-IP form? A lot of the time it happens unintendedly if you make some trivial typo in a "Deny from ..." line. It only takes a single error--a superfluous comma, say--to throw the whole log into resolve-IP mode. (It has a correct technical term which I can never remember.) Some users like this format; most don't, as if makes logs harder to read. In fact Apache [httpd.apache.org] (docs mysteriously for 2.5, which doesn't really exist) recommends not doing it:

If HostnameLookups is set to On, then the server will try to determine the hostname and log it in place of the IP address. However, this configuration is not recommended since it can significantly slow the server. Instead, it is best to use a log post-processor such as logresolve to determine the hostnames.

And that's if you have set it intentionally; as noted above, it can also happen by accident. (Ask how I know :()

[TorontoBoy]

My last major xmlrpc.php attack was in 2016. It was at 57 posts, long but uneventful. I could not track down the cause or persons. Yes, this is specific to WP. Here is a snippet:

108.175.6.* [11/Dec/2016:21:38:18 POST /xmlrpc.php HTTP/1.0 200 403 http://example.com/xmlrpc.php Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; Touch; rv:11.0) like Gecko

Specifically for your case, did a HOST protected2.by0x6ab.com and came up with nothing. by0x6ab.com is available for sale, so it is faked. Mr. G comes up with nothing.

Why would the IP address come up with the host name? Are you sure there is not an error in your htaccess, and your server is converting to host names instead of IP addresses? This has happened to me...a couple of times...not good.

Ahem. Your htaccess has some error and your server is now returning host names. I did a "host 185.244.43.71" and it came back with a somewhat familiar name:

$host 185.244.43.71
71.43.244.185.in-addr.arpa domain name pointer protected.by0x6ab.xyz.

Here's some code for your htaccess, if you wish, bit I did not:
# Block WordPress xmlrpc.php requests

<Files xmlrpc.php>
order deny,allow
deny from all
allow from 123.123.123.123
</Files>

Currently all my POSTS need a request header lang, which bots usually are devoid, so receive a 403.

[jackson]

Wow people, appreciate all your replies and suggestions. Kind of overwhelmed - in a good way of course and not the same ways as my xmlrpc.php getting dished.

That said, let's look at what we have here:

@ not2easy - thanks for your suggestions in your first post. Yup, picked up that UnionGroup were the perps. Was even nice enough to send them a message - to their abuse@ address. Dodn't expect a reply and perhaps, in the process, turned a red flag into a green one? As for the said xmlrpc.php, do I really need it? Seems not and, after this, may just delete it. More on this later.

@ lucy24 - have to smile every time I read your replies and comments. The term "acerbic" comes to mind - said in jest of course. That said and in my haste, I may have screwed up something in my .htaccess file - as in:

It only takes a single error--a superfluous comma, say--to throw the whole log into resolve-IP mode

What I did was to put all the IP addresses on one line and separated them by commas. Not a good idea or, so it seems? However, in doing that and activating the .htaccess file, this is when that "switch" occurred - changing from IP address to the faked URL. Hence my post. We live and learn and, in the process, live to see another day.

In all this, will follow up on what you have suggested. So, thanks for all that.

@ TorontoBoy - As metioned to my reply to lucy24, seems like I've screwed things up in my .htaccess file. So, thank you indeed for your suggestion. I'll implement this when I'm done here.

As for this xmlrpc.php attack, it went on for 2 days if not longer. As a result, the various affected log files swelled to several megabytes, the MySql database swelled by several 10 megabytes, CPU usage shot up to a sustained 50%+ and frequently hit a sustained 80% - my notification level. This attacked also overwhelmed my current implementation of fail2ban - as in shutting down the service. Now need to do some more worked on that one.

Now, here's an interesting question for everyone [and could perhaps call for a new thread], why are you implementing the Apache 2.2 syntax and not the Apache 2.4 syntax? Referring here to

order deny,allow, etc

instead of

Require all denied

In 2.2 that set up always confuded me [yup, as in "confused"]. It was always like a juggling act. Not being a good juggler, dropping one of those deny/allow balls and the whole directive gets screwed up. With Apache 2.4, it's now all there - as in yes or no - in just one line. Your insights would be appreciated.

@ all - Regarding this whole experience, under other circumstances, this could have been "devasting". As it is, I'm still in the process of setting up this VPS and the websites it will support. As suggested in the opening of my message, it's been "fun". "Fun" in not only experiencing this kind of stuff first hand now but also getting back into the whole CLI [Command Line Interface] thing and getting down to the "metal" or "bare bones" as they say - as in dealing directly with Apache and all the other Linux based services - BIND, PostFix, Awstats, etal. No more shared hosting and having everything hidden by cPanel or whatever. Tried Webmin at one point and dumped that - just for good measure.

All that said, here are some backgrounders:

This is my replacement xmlrpc.php file. I "gutted" the original file and replaced the contents with the following:

<?php
/* Redirect browser */
header("Location: http://127.0.0.1");

exit;

?>

I found this solution on the net. And, as the poster suggested, "if you want to have a little fun". I guess the caveat here is, "Try this at home" and not on your production server. Like others no doubt will, I had to try this out to see how it worked. And, it was fun while it lasted.

As for my .htaccess file, in my haste, I had this one line:

Deny from 193.201.224.218, 185.244.43.71, 185.244.43.86, 185.244.43.88

Now, on to mend things as per the suggestions on this thread. As a BTW, my WP .htacess file still has its attribution to Jim Morgan - yes, from that time back. As with Apache 2.4's suggestions, may soon move all the .htaccess directives "in-house" - as part of the virtualhost config file. Kind of depends on what they come up with in WP 5.0? Ehh, all part of a new learning curve...

Thank you all.

[edited by: phranque at 6:14 am (utc) on Dec 1, 2018]
[edit reason] unlinked url [/edit]

[not2easy]

why are you implementing the Apache 2.2 syntax and not the Apache 2.4 syntax?

Believe it or not, there are still hosts on Apache 2.2 today. And I am talking about some well known, large hosting companies. I am using 2.4 today but have not made significant changes to my .htaccess files because they are still doing what I expect them to do. I guess it is the old "if it ain't broke, don't fix it" - that and being much more familiar with what I'm using.

I am aware of the new syntax, but until I need to do something like rewrite a pile of .htaccess files, I'll do something that I prefer to do instead.
_{(which is almost anything else)}

[lucy24]

What I did was to put all the IP addresses on one line and separated them by commas. Not a good idea or, so it seems?

Been there. Done that. Fine-tooth-comb the file, deleting all commas. The syntax is simply
Deny from 1.2.3 4.5.6 7.8 9
with as many arguments as you choose to put in each line. The space is the only required separator. (This is 2.2 mod_auth-thingy syntax. Most 2.4 installations will have mod_access_compat so you've got time to change over to 2.4 syntax; things won't break in the meantime. For 2.4 you will of course say Require ip [httpd.apache.org] instead, but the IPs get listed in the same way.)

[jackson]

Hi,

Regarding 2.4, I hear where you're coming from - as in the, "if it ain't broke, no need to fix" thing. As for the legacy stuff, been there and done that. Heavens, I've been lurking around these parts [WebmasterWorld] for the past 15 years or so - if my profile is anything to go by!

As mentioned in my posts somewhere, I've just recently "transitioned" from shared web hosting [since that time] to managing my own VPS. In the process, have sort of made an "executive decision" [as far as "one man bands" go] to pick up on the "latest and greatest" in whatever is out there and working with that. With the pending release of WordPress 5.0, this maybe something a game changer. This being the case, kind of makes senses to work with 2.4 - this to sort of "future-proof" things.

The thing that I am finding attractive about 2.4 is the "virtual hosting" syntax and file structure. A kind of separation of config files and the move away from the dependence on what was usually a single or group of httpd file/s - this as I understand it. Added to this, things like SSH, SSL and Let's Encrypt has changed the landscape some. This new 2.4 syntax seems to help simplify things and to better manage these features. As mentioned, I'm still standing at the bottom of the "grease pole" in this regard - as in still having a lot to learn.

All that said, it's been kind of hard to get away from the 2.2 legacy, this in terms of documentation and references. With 2.2, one is spoilt for riches. With 2.4, things are still decidedly sporadic and porous.

[jackson]

Hi All,

Here's something of a wrap up of the above issue. With the good advice offered on this thread and elsewhere across the net, we seem to have got something sorted. As a step by step thing, we did this:

1. Completely removed the xmlrpc.php file from the WP installation.

2. In my .htaccess file, made these modifications:

# BLOCK IPS
<Limit GET POST>
order allow,deny
deny from 193.201.224.218
deny from 185.244.43
deny from 185.244.40.0/22
allow from all
</Limit>

# Block WordPress xmlrpc.php requests
<Files xmlrpc.php>
Require ip 111.222.333.444 [put in your own service's IP address]
</Files>

[C'mon people, in that files block, you gotta love that single line of code - this regarding the Apache 2.2 vs 2.4 topic? A wind up of course.]

3. The thing that seemed to nail it - as in scrubbing this muck off my system's log files, was correctly setting up the fail2ban app. In fail2ban - /etc/fail2ban/filter.d - I created an xmlrpc.conf file. In there, placed the following piece of code:

[Definition]
failregex = ^<HOST> .*POST .*xmlrpc\.php.*
ignoreregex =

This piece of coding came courtesy from an internet resource.

In the fail2ban jail.local config file, added this directive:

[xmlrpc]
enabled = true
filter = xmlrpc
action = iptables[name=xmlrpc, port=https, protocol=tcp]
logpath = /var/log/apache2/access.log
bantime = 43200 ; 12 hours
maxretry = 2
findtime = 7200

As something of a rookie's explanation, the said transgressing IP address get's logged in the iptables app. This after they've tried to access the xmlprc.php file 2 or more times and this with a 7200 second [2 hour] period. In making this transgression, the said IP address gets banned for 12 hours. In these settings, I'm perhaps being a little too generous here? Time will tell.

Moves are a foot to develop a better understanding of fail2ban, its set up and the associated iptables app and looking to build a permanent bad IP ban regime.

As a final word, with the xmlrpc.php ring fenced like this, I may look to reinstate the file - not that it seems like I need it, not at the moment at least?

So, again thank you all for chiming in and for all your comments and suggestions. Here's hoping the above will be of value to others.

Take care...

[lucy24]

A wind up of course.

I sure hope so :) When I was looking up the correct name of mod_access_compat, I found some blahblah from Apache saying they don't recommend combining 2.4 and 2.2 access directives, even if it's technically possible. They may even have uttered the dreaded “unintended consequences” line, which is Apache-speak for “the world as we know it will come to an end”.

[JamesSC]

The xmlrpc.php is used to allow WP admin (and any users that have permissions set) to write posts remotely, without logging in to WP which is why it is such a target. It is only used remotely.

It is an option which imo is only used by WP users

not2easy: xmlprc.php is also a file the Automattic WordPress plugin Jetpack absolutely must have access to in order to function (yes, I'm aware of Jetpack's appetite). Typically one will insert a rule in .htaccess to deny access to xmlprc.php to anything other than Jetpack.

[not2easy]

Yes, Jetpack users have to keep their xmlrpc.php file available, and for WP sites that started at wordpress.com, it is almost always a de facto plugin. This is an example of why I hesitate to suggest fixes for anything WP without knowing what tools are part of the environment. The simple fix can break something else if you are not aware of all the plugins that are in the mix.

[lucy24]

Typically one will insert a rule in .htaccess to deny access to xmlprc.php to anything other than Jetpack.

But would this translate into a direct (external) request for the file? It sounds like another of those situations where a %{THE_REQUEST} condition will nicely fit the bill.