Forum Moderators: phranque
These Web sites are compromised at the server level; All virtual hosts on the server will become malware downloaders. Servers equipped with mod_bandwidth_limited and mod_dl seem to be vulnerable.
Having several sites that don't use external JavaScripts, or where the JS is small enough to include in-line, I decided that even if my servers are compromised, I didn't want to play any part is spreading this malware. So, I put a simple one-line mod_rewrite patch into my .htaccess file:
RewriteRule \.js$ - [G]
For sites that do use a small number of external JavaScripts, the nature of this malware implementation indicates that using a 'valid filename filter' may be helpful:
RewriteCond $1 !valid_js_script1_path\.js$
RewriteCond $1 !valid_js_script2_path\.js$
RewriteCond $1 !valid_js_script3_path\.js$
RewriteRule ([^.]+\.js)$ - [G]
Jim
At any rate, if you have no external JS scripts --or only a few-- these simple solutions should at least prevent the payload installer script from downloading from your server if your server gets infected.
Jim
As far as viewing the request in your stats, yes, you could do that too ... but I would rather opt for immediate notification and be able to analyze details I captured from the request as opposed to only that information captured in the access and error logs. The important part here is the timeliness.
At the moment a bad js file is anything that matches [a-z]{5} but since they have root on your machine they could easily flash update all the machines to serve the exploit in a different file or using a totally different method. Its best if you know ASAP that something might be wrong so you can fix the root cause.
[cpanel.net...]
Kudo's to them for this - I have seen them wrongly accused as the source of the compromise.
The fixes above are unlikely to work. This thing overwrites httpd in memory, and does not log these requests. I highly doubt that it will get to mod_rewrite in the module list, and it certainly won't show up in your stats.
The only 'tracking' that you could do, woudl be to do a periodic tcpdump for the requests as is described on the cpanel page.
This thing is very clever and well implemented. It will only serve the js once per IP/domain, and nothing shows up in the access log. It is affecting fully up-to-date boxes, and I suspect using a yet-unknown vulnerability to get root access.
How might I get infected with this malware
The scary thing about this, is that no one knows that yet, despite the fact that it has infected 10's of thousands of sites.
how would I detect if my pages have been injected with it
One is that the first request you make to your site may or may not have a <script> tag inserted into it. IF it does, it will only have it on the first request your ip makes. Other than that, doing a tcpdump as described on the cpanel page, or booting to safe mode with a system rescue CD and looking for modified files.
One simple but not 100% reliable thing you may do, is go to the page with IE, and look for an activex warning that you know shouldn't be there. Again, this will only be there for the first request of the IP, and then only randomly.
The cPanel security team also recognized that a majority of the affected servers come from a single undisclosed data-center.
the cPanel security team believes that the attacker has gained access to a database of root login credentials for a large group of Linux servers.
The cPanel security team asserts that the Boxer variant includes a small web-server which is how the Javascript is distributed to unsuspecting users of any website on the server. It is believed that the Javascript include is injected into the HTML code after ApacheŽ has served the file but before it has traveled through the TCP transport back to the user of the website. The web-server is not loaded onto the hard drive directly but loaded directly into memory from the infected Boxer binaries.
The fixes above are unlikely to work.
Actually, I believe they will, from what I understand about this so far.
This thing overwrites httpd in memory, and does not log these requests. I highly doubt that it will get to mod_rewrite in the module list, and it certainly won't show up in your stats.
It loads it's own httpd server in memory, yes, but from what I understand only to inject a Javascript include into your HTML after your Apache server has finished it's job and sent the file out. After the httpd-in-memory-server finishes up it's little edit, it pushes the page on through to the end user. Then the end-user compromise starts. The included Javascript file is requested from your host ... all the Javascript examples I have seen so far show that the path to the malicious Javascript is on your server, the root-compromised host. Your Apache server looks up that file and sends it off, just like any other request, right on through mod_rewrite, etc. Therefore the request should still show up in your logs. The only thing is, it is a one time deal. You are only going to see a request for this particular js file once. After you dish up the malicious Javascript, the end user is redirected to the *bad* site(s) and it could very well start getting ugly for your end user.
I have indeed read that there is an httpd server in memory but the only information I have read so far is that the requests are being modified on the way out, not coming back in. The compromised user's IP address is stored on the server so the code checks against that file before modifying the HTML being served again. If the IP is found, the HTML is not modified this time before being passed along.
If there is any information to the contrary I have yet to review it.
Most exploits are easily identified, but the lengths the creators of this exploit went to to obfuscate it are impressive.
If only these people would use their skills for something other than making other people's lives a nightmare and making some cash...
The vast majority of affected systems are initially accessed using SSH with no indications of brute force or exploitation of the underlying service. Despite non-trivial passwords, intermediary users and nonstandard ports, the attacker is able to gain access to the affected servers with no password failures.
What it looks like is that information only known at the provider's level is used to gain access to the system. Or there is a major vulnerability in the SSH server which is not discovered yet.
Although I appreciate jdMorgan's efforts to minimize the current effects by reconfiguring Apache to not allow JS files, or at least restrict their use to a whitelist of trusted filenames, this will only be a temporary solution. According to the cpanel article the attackers seem to have easy root access to the infected systems and have infected a number of system files which may contain additional backdoors for them to install enhanced versions of the software.
To prevent this kind of attack you should take other measures IMHO.
lammert said
Uninstall your control panel software
Kaled.
I am not saying that a control panel is a security threat itself, just as httpd, mysqld, sendmail and all the other packages I mentioned in my post are no security threat by themselves. But they are all an extra hook that hackers can attach to and may try to exploit. Reducing the number of doors to your server, and using only doors that are not feature rich, like a SSH shell server with only protocol 2 allowed with key authentication and limited allowed IP addresses reduces the risk that a hacker finds a hole.
I know that no system is totally hacker proof, and even if it was, it could still be compromised by compromising systems that are in the trusted environment of the hacker proof server.
For example, I don't want to think about the situation where a hacker manages to upload a new openssh version to one of the redhat, centos, fedora, debian, ubuntu or any other linux clone repository.
I think several points need to be emphasized for webmasters and users (rather than server administrators)...
Kaled.
Reducing the symptoms in the period that the real source has not clearly been detected yet is good because it may reduce the speed at which it spreads and reduce damages, but the only effectrive solution will be to close the holes the malware used to infect the current webservers.
There are two points not discussed in the finjan report (unless I missed them) that server admins should consider...
Kaled.