|Securing A Linux Web Server|
A basic overview for the co-lo or self-managed
| 12:18 pm on Mar 15, 2007 (gmt 0)|
If you self-manage or co-lo a Linux server, you absolutely need to know how to secure a server. If you don't know, you need managed hosting. Full stop. No questions.
Here's a (very) basic outline for you. You'll need to do some research on each step. This post is specific to a Linux server running Apache and PHP, a very common combination. Servers running ASP or other software will no doubt require other considerations.
Never ever use standard ports or your main IP for shell access, and never use FTP.
Result? Every server requires at least two IP addresses - one for the httpd server that resolves from your web domain, and one for SSH (running on non-standard port). So we have:-
www.example.com A RECORD 220.127.116.11 (httpd binds to this)
.... and one further IP address, with no reverse lookup set in DNS for SSHd and for SSHd only. And not on port 22 - go for a random port > 1024.
What about FTP? What about it indeed. You don't need it if you have SSH - any half decent FTP client can tunnel in over SSH. It's called secure FTP or SFTP. FileZilla will do it.
Mount /tmp directories with noexec and nosuid to prevent download/execution.
Note : httpd will still be able to write and execute anywhere the user httpd has privileges. Think it through and make double sure.
Disable unecessary functions
This can cause problems for sites that use them. Do one at a time and test.
Disable GCC, WGET and CURL
If you do nothing else, do this. Assuming your site doesn't need them. In fact, here's a really really obvious one - anything your site doesn't need, disable it.
Here's an example of an old ubbthreads hole:-
20ownz.txt;perl%20php.txt HTTP/1.1" 200 31785 "-" "LWP::Simple/5.63"
So, we have:-
.... from one single http request. 5 scripts just got executed. If Apache didn't have the ability to use wget, this would have been averted.
Priviledges and Groups
Make priviledge groups or no exec them and regrant on an as and when needed basis.
What are we running and why?
Disable unnecessary servers. PS AX is your friend.
You might consider to leave telnet running in case SSH fails, but only where the co-lo or self-managed hosting co. has no facility for remote KVM in an emergency. Otherwise kill it.
Are we up to date?
Apply patches. That's really easy to do. Absolutely no excuse.
mod_security and mod_rewrite
People here often think of mod_rewrite only in terms of SE friendly URL's, but it's actually a fantastic way to hide your network and application topologies. Think about it, I can take a PHP site and make it look like it's running on ASP. That's pretty cool. I could even have Linux Apache Server claim to be a Windows Apache Server. That's pretty cool too.
If you don't mask it, switch it off. Apache defaults to telling the World all about itself. That's bad - edit the httpd.conf file to remove any broadcast of installed modules, version numbers etc etc
You must must must know how to use IPTables. Forget any of the web based interface rubbish. Learn how to work it at the command line. If you can't do that, get a managed server (seriously).
Ground up re-write your IPTables rules. You only have open what you need.
Have we been rooted? You need to do this immediately in view of what you have just experienced.
Root check kits:-
Here's the rub; I just blasted through this in about 10 minutes flat. It requires a lot more thought and implementation, as you probably guessed. If you didn't guess, and you try and read this as a how-to guide, then pack it in and go buy yourself a managed server. If you read through this and go "ahhhh.... OK, I get it" then you have enough to go and do some research, buy a decent book on securing a Linux server and go to it.
From scratch, with no experience, you could learn enough and get the job done in 24 hours, assuming your basic Linux skills are up to scratch.
<Edit>Fixed Links as per Henry0</Edit>
[edited by: trillianjedi at 1:53 pm (utc) on Mar. 19, 2007]
| 7:39 pm on Mar 15, 2007 (gmt 0)|
Excellent post, TJ!
I'd add installing the Bastille-Linux [bastille-linux.sourceforge.net] server hardening system to that list.
chkrootkit is great for detecting breakin attempts and to perform forensics on a successful attack, as are logwatch [logwatch.org], mod_security and the Sentry Tools [sourceforge.net] suite of programs (PortSentry, HostSentry, LogCheck).
For DoS/DDoS protection, I'd recommend using mod_evasive [zdziarski.com] by Jonathan Zdziarski.
Lastly, make sure that the SELinux [nsa.gov] features of your server have been initialized and are available for you to use (it's part of the Linux kernel since v.2.6). Modern Linux distributions (i.e. Fedora Core 4+) include that version of the kernel or better.
A Linux server is just another server until it is made as secure as possible. Of course, it starts out a little more secure than other types of servers, but it needs more protection than it comes with, usually.
[edited by: encyclo at 1:26 pm (utc) on Dec. 13, 2009]
[edit reason] updated link [/edit]
| 8:38 am on Mar 18, 2007 (gmt 0)|
Thanks trillianjedi. I guess I need managed hosting. :) I'm not sure I can learn all these stuff.
I'm now with <snip>, I thought they have managed hosting, but they mean another thing by "managed". Could you tell me few examples of "real" managed hosting who will secure server for me?
By the way, do I need close all these holes if I will choose Windows Server, or software updates will be enough?
[edited by: trillianjedi at 1:05 pm (utc) on Mar. 18, 2007]
[edit reason] No specifics please, as per our TOS... [/edit]
| 1:04 pm on Mar 18, 2007 (gmt 0)|
I've split this thread over to here as it's perhaps more suitable for the Linux forum.
With any server, Linux, Windows, Sun, Mac OS etc - you need to consider security.
Recommending specific hosts is against our charter I'm afraid - we could never tell whether a recommendation was by a member that actually owns, or works for, the recommended host.
| 8:25 pm on Mar 18, 2007 (gmt 0)|
Here's another perspective on security with some good suggestions: How to secure UNIX server from hackers? [webmasterworld.com]
| 8:34 pm on Mar 18, 2007 (gmt 0)|
Nice summary. Another step I'd recommend is to set up SSH login keys and disable SSH password logins.
buy a decent book on securing a Linux server and go to it.
Buy the O'Reilly book "Building Secure Servers with Linux"
| 10:05 pm on Mar 18, 2007 (gmt 0)|
Simply awsome post, this one's going streight into the bookmarks. When it comes to a server you can never to to safe, this will certainly point people in the right direction.
| 5:19 am on Mar 19, 2007 (gmt 0)|
Great topic to post about!
...here's something you need to do on top off all the other security measures...
Backup Backup Backup and Backup
If your server is compromised and attacked in any way, the best line of defence is to be able to say "ok, wipe the drive" and switch to a backup server. You never want to be in the position of having to copy things off an infected machine or having to try to fix an infected machine with unknown damage.
Here's a tip regarding updates - if your server has 'local' repos for use with yum and/or rpm - check they are updated daily, or switch to pulling them from somewhere outside.
| 9:18 am on Mar 19, 2007 (gmt 0)|
Changing the SSH port or a dedicated ip just for the ssh server are not good protection. Changing ports or ips might give you a bit more safety but someone who really intends to hack your server is not stopped by a different port or even a different ip.
It does save you some bandwitdh from all the scanners though (I have on average about 500 attemps to login through ssh on my servers). But these will only be a real problem when you have extremely weak passwords, like apache/apache or backup/backup.
So my addition to this thread:
Do not give customers shell access by default. If they ask for it, give them a chrooted shell wherever possible. Customers tend to have very weak passwords!
| 9:50 am on Mar 19, 2007 (gmt 0)|
Rkhunter is also very useful for detecting unwanted intrusions...
| 10:46 am on Mar 19, 2007 (gmt 0)|
|Changing the SSH port or a dedicated ip just for the ssh server are not good protection. |
You misunderstand the purpose of the secondary IP perhaps. It's an additional obstruction to the would be hacker that wants to hack www.example.com. He can't just SSH to "www.example.com", or to the IP that resolves to. He has to find a second IP, and not having any DNS setup for it removes the obvious guesses such as ssh.example.com:22.
Never ever run anything that's for admin only access on standard ports. Ever. And always run SSH on a dedicated IP, or, if you only need access from one machine, blocking all access to SSH other than from your IP address is even better.
To be able to hack a server, you first of all have to find that server.
| 11:38 am on Mar 19, 2007 (gmt 0)|
I like the idea of running ssh on a seperate IP address.
The ultimate defence though is to turn it off when not needed. If you only ssh in for 10 minutes a day to check something why let it run for the other 1430 minutes?
OK, some sites may need it running 24/7 but there's probably a lot of sites which don't and would be wise to shutdown the ssh server when not in use.
The best way to do this IMO is to use email and a procmail recipe to swtich on and off the services.
I posted on a similar thread about webmin. Just setup a recipe to detect a particular user and subject like
ssh OFF or ssh ON
When you then need to access the server via ssh you just email email@example.com with the appropriate header in the subject.
| 12:06 pm on Mar 19, 2007 (gmt 0)|
Looks like your links (exec and popen) should read US2 and not US4
For some reason I cannot open them otherwise.
| 1:44 pm on Mar 19, 2007 (gmt 0)|
Regarding ssh: always make sure the option "PermitRootLogin" in the sshd_config file is set to "no". That's one potential attack vector less.
| 3:30 pm on Mar 19, 2007 (gmt 0)|
|OK, some sites may need it running 24/7 but there's probably a lot of sites which don't and would be wise to shutdown the ssh server when not in use.[snip] |
IMHO techniques like this just set you up for trouble later. Great idea in theory, not sure how good it is in the real world.
Would you want your locks on your front door to only function between 7am and 8am, and between 5pm and 7pm? What happens if you leave something at home and need to call back for it at lunchtime?
If your webserver stops responding mid-morning, do you want to have to wait until the designated 'ssh-time' to get in to fix the problem?
Personally, I'd suggest configuring ssh to only accept public key authentication, and leaving ssh open all the time. The password-guessers can grind away all they like on my servers if the only way in is to use my private key. Saves remembering a complex root password too.
For those who are completely paranoid, you should disable ssh completely and access your server via serial console redirection provided by your ISP....
...you DO HAVE a serial console configured, don't you? They've saved me more than once. Far more useful (and more secure) than an extra ip address for ssh.
| 3:39 pm on Mar 19, 2007 (gmt 0)|
webdoctor, I suggested a 'request for access' type method.
You send off an email requesting the service. Only then is it started.
I had thought of using a webform to control services but that would introduce new insecurities as php or whatever would need functions which are disabled. And this I had already touched on with the Webmin thread. I don't use this everyday but it was running 24/7. The email method allows me to switch services (webmin, ssh) off and on at will from my mobile etc.
"The security deflector shield will be deactivated when we have confirmation of your code transmission." :-)
[edited by: Frank_Rizzo at 3:40 pm (utc) on Mar. 19, 2007]
| 7:04 pm on Mar 19, 2007 (gmt 0)|
"Port knocking" (see e.g. [linuxjournal.com...] ) is another technique for "hiding" sshd (and other) services.
| 8:54 pm on Mar 19, 2007 (gmt 0)|
|I suggested a 'request for access' type method. |
You send off an email requesting the service. Only then is it started.
...and if your mailserver has problems, you lose control of your server. I wouldn't want to be in *that* boat without a paddle.
I'd prefer a serial console connection. Never heard of any serious software issues with a properly screwed-in serial cable :-)
|I had thought of using a webform to control services (...) |
I prefer to use the command line to control services. After all, that's what it's there for
Here's a question: what is it about running a ssh daemon in particular that worries people?
Software vulnerabilties in the ssh daemon? Come on, you're running a public web- and/or mailserver on your box, it's not as if your machine is invisible to the internet. You're probably running a few scripts, PHP, cgi, MySQL, ...
If you're worried about brute-force password attacks, either
1) choose a long password (my root passwords tend to be 10 - 15 characters long, and they're random. Yes, *really* random. (Of course they're written down on a piece of paper, but if my house gets burgled *and* the burglars crack open my safe then I think losing my root password is probably the last of my worries...)
2) switch to public/private key authentication. Just make sure you back up your private key properly - it's a long drive to the data center :-)
|[Port knocking is...] another technique for "hiding" sshd (and other) services |
IMHO there's (a) a whiff of security by obscurity about this, and (b) it's also just a bit too "James Bond" for me. I'd prefer to use serial console redirection.
Another question: How many people here block ping requests at their firewall? Why?
| 8:29 pm on Mar 23, 2007 (gmt 0)|
Regarding (any) FTP server: Start it only when you really need it. Stop/kill it when finished.
| 1:08 pm on Mar 26, 2007 (gmt 0)|
>>Another question: How many people here block ping requests at their firewall? Why?
I think the purpose here is basically, if you don't need it, turn it off. Does anyone else need to ping your servers? If not, turn it off. If you need to ping your servers,you can always turn it on temporarily.
I sometimes use webmin. But I turn it on, use it, then turn it off again. Same idea.
| 9:34 pm on Mar 29, 2007 (gmt 0)|
Is it safe to enable SSH (Shell Access) for all accounts in cPanel? Is is safe to use SSH1, or SSH2 is better? Is it safe to store SSH password on my home computer (Mac, Fugu SSH)?