homepage Welcome to WebmasterWorld Guest from 54.167.179.48
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Hardware and OS Related Technologies / Linux, Unix, and *nix like Operating Systems
Forum Library, Charter, Moderators: bakedjake

Linux, Unix, and *nix like Operating Systems Forum

    
VPS memory allocation
barns101

5+ Year Member



 
Msg#: 4018895 posted 11:58 am on Nov 4, 2009 (gmt 0)

I have a VPS that is experiencing frequent Apache crashes. From what I have been able to work out, it looks like it is running out of memory.

Commands such as "top", "ps" and "free" only show memory usage for the server as a whole, but I found the following script that will show memory usage for my VPS alone:

#!/bin/bash
bean=`cat /proc/user_beancounters`
guar=`echo "$bean" grep vmguar awk '{ print $4;}'`
burst=`echo "$bean" grep privvm awk '{ print $5;}'`
alloca=`echo "$bean" grep privvmpages awk '{ print $2;}'`
priv=`echo "$bean" grep physpages awk '{ print $2;}'`
let guarenteed=guar*4096/1048576
let allocated=alloca*4096/1048576
let used=priv*4096/1048576
let burst=burst*4096/1048576
echo "VPS memory usage:"
echo "Allocated: $allocated MB"
echo "Used: $used MB"
echo "Guarenteed: $guarenteed MB"
echo "Burstable to: $burst MB"

Running this script shows that I have 200MB guaranteed memory, burstable to 400MB. It also says 284MB memory allocated and 141MB memory used.

Please could someone explain the difference between the 284MB allocated and 141MB used?

 

webdoctor

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 4018895 posted 5:19 pm on Nov 7, 2009 (gmt 0)

Sorry not to answer your question directly, but have you considered looking at httpd.conf to check whether Apache is tuned to run on a server with relatively little memory?

Settings you might like to check include StartServers, MinSpareServers, MaxSpareServers, ServerLimit, MaxClients and MaxRequestsPerChild.

I have one very low spec VPS which wouldn't run Apache at all - until I adjusted the settings listed - now it works fine.

barns101

5+ Year Member



 
Msg#: 4018895 posted 7:13 pm on Nov 7, 2009 (gmt 0)

Hi, thanks for the reply. :)

Optimising my Apache setup is one of the things on my "to do" list. Off the top of my head I remember reading about reducing KeepAliveTimeout and a couple of other things, but the material is on my PC in the office.

Can you recommend any values for the settings that you mentioned that would be suitable for a VPS with little memory?

webdoctor

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 4018895 posted 9:20 pm on Nov 7, 2009 (gmt 0)

Can you recommend any values for the settings

Can you post your current settings? How many visitors/pages is your server handling, and what else is running on this VPS (databases, mailservers, ...)?

"Optimising" usually implies something works reliably, but that performance can be improved. If your server is crashing frequently then I wouldn't call this "optimising", but "making it work"

For reference, my baby VPS has these settings

StartServers 1
MinSpareServers 1
MaxSpareServers 3
ServerLimit 15
MaxClients 15
MaxRequestsPerChild 0

but this server runs a *very* small site, so these values are unlikely to be appropriate for your site.

On the other hand, my server has an uptime > 300 days (it's pretty stable) so perhaps you should try these settings? :-)

Are you using sysstat (or similar) to log the load on your server? Might also be worth checking out Apache's mod_status module too.

barns101

5+ Year Member



 
Msg#: 4018895 posted 1:03 pm on Nov 9, 2009 (gmt 0)

Thanks for getting back to me.

The VPS is running the Plesk control panel with a LAMP setup and Spam Assassin installed as well.

Current Apache settings are as follows:

KeepAlive Off
...
<IfModule prefork.c>
StartServers 1
MinSpareServers 1
MaxSpareServers 5
MaxClients 10
MaxRequestsPerChild 0
</IfModule>
...
<IfModule worker.c>
StartServers 1
MaxClients 10
MinSpareThreads 1
MaxSpareThreads 4
ThreadsPerChild 25
MaxRequestsPerChild 0
</IfModule>
...
<IfModule perchild.c>
NumServers 5
StartThreads 5
MinSpareThreads 1
MaxSpareThreads 4
MaxThreadsPerChild 20
MaxRequestsPerChild 0
AcceptMutex fcntl
</IfModule>
...
HostnameLookups Off

I can't find any reference to ServerLimit.

The VPS has 30 websites, most of them are fairly low traffic, but the busiest 5 websites served 360,000 pages last month. I guess the other websites might take that to a total of 450,000 - 500,000 pages.

At the moment I have to admit that I'm taking a reactive approach rather than a proactive one because I'm a web developer, not a server admin and these responsibilities have fallen on me regardless! This is the first time that we've had server problems so I have set up a bash script that's run by cron every 5 minutes to check whether Apache is running, and restart it if not.

webdoctor

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 4018895 posted 3:57 pm on Nov 10, 2009 (gmt 0)

Q: Do you know which Apache MPM your server is using?

You've got configuration settings for Prefork and Worker, your server is using one or the other, not both...

barns101

5+ Year Member



 
Msg#: 4018895 posted 4:47 pm on Nov 10, 2009 (gmt 0)

Thanks for your continued help. :)

Apache is using Prefork.

bkeep

5+ Year Member



 
Msg#: 4018895 posted 10:32 am on Nov 12, 2009 (gmt 0)

Has this issue been sorted? I would be curious to find out what other things are running, users cron scripts, some mail script choking out the server, backups? 500,000 pages per month seems pretty low volume so unless you get all those in a span of ten minutes I wouldn't think that to be the cause.

I am no Apache guru. I run a small server as well and what I have found regarding general info related to Apache optimization.

MaxClients - Total number of concurrent connections.
Locate it in the configuration file. This should be set to a reasonable value. I suggest using this formula to determine the right value for your server. So for example if you have 2 GB or RAM set this value to 300.
There is no reason for you to set it any higher unless you have a specific problem with this value. A high value can lead to a complete server hang in case of a DOS attack. A value too low can create timeout problems for your clients if the limit is reached.

baseline to start it is usually recommended MaxClients = 150 x RAM (GB) so in your case 150*.25 so around 35

MinSpareServers and MaxSpareServers - MaxSpareServers and MinSpareServers control how many spare (unused) child-processes Apache will keep alive while waiting for more requests to put them to use. Each child-process consumes resources, so having MaxSpareServers set too high can cause resource problems. On the other hand, if the number of unused servers drops below MinSpareServers, Apache will fork (an expensive operation) new child-processes until MinSpareServers is satisfied.

Leave those values to:

MinSpareServers 5
MaxSpareServers 10

My config is a little different since I run a cpanel server but I would also look into the other things that may be running. Spamassasin can suck up memory too not leaving any for Apache or anything else. What kind of loads do you get when you are running out of memory? One last question what shows up in the error log?

[httpd.apache.org...]

barns101

5+ Year Member



 
Msg#: 4018895 posted 3:23 pm on Nov 13, 2009 (gmt 0)

Thanks for responding. :)

No, the issue is still ongoing, I'm still seeing intermittent crashes. It's got a lot better recently, probably because the busiest website has seen a massive drop in traffic due to its seasonal nature.

As far as I am aware there are no cron scripts, mailers or backups running. (We do our backups early every morning and they don't seem to cause any problems.)

MaxClients is only 10, so that should be OK I would have thought. My MinSpareServers and MaxSpareServers are 1 and 5, respectively. So you recommend raising those values? (I read that values of 3 and 10 could be optimal on a small VPS, so that would sound about right.)

We are running Spam Assassin and I have decreased the number of worker processed from 5 to 2. It's hard to tell whether it's made much of a difference.

When I see the VPS running out of memory I usually just restart Apache to fix the problem. However, as far as I recall the load isn't any higher than usual.

I can't see anything in the Apache error log that means much. It's mostly errors like this one:

[...date...] [error] (12)Cannot allocate memory: fork: Unable to fork new process

bkeep

5+ Year Member



 
Msg#: 4018895 posted 8:28 pm on Nov 13, 2009 (gmt 0)

I think your error corresponds to this statement "Apache will fork (an expensive operation) new child-processes until MinSpareServers is satisfied".

I would try that, 3 or 5 should be enough to notice a difference. I would start at 3 and adjust from there if needed. Also how prohibitive is it for you to upgrade your VPS to a plan with a higher memory allocation? As far as I know VPS machines don't have swap space (someone correct me if I am wrong) so when you reach your memory limit processes just get killed off and can result in ugliness.

You may also want to look into your loaded Apache modules and comment out any that are not needed.

barns101

5+ Year Member



 
Msg#: 4018895 posted 9:34 pm on Nov 13, 2009 (gmt 0)

Thanks, I'll try increasing MinSpareServers. :)

I've got a load of changes that I've seen recommended specifically for Apache running on VPSes, I just need to find the time to start tweaking! (Removing modules is one of the suggestions.)

I don't think upgrading the VPS is feasible, although we could move the busier websites to other servers if things don't improve. Ideally I'd like to be able to tune the VPS and learn from this rather than taking the easy option of moving the websites! ;)

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Hardware and OS Related Technologies / Linux, Unix, and *nix like Operating Systems
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved