Help with Load Average - linux guru?

Site is bogging down with CPU idle!

[spagmoid]

I understand that Load Average is the average number of processes in the run queue. My site activity has grown so that I get 100-150 httpd processes active at once. Last night the LA went up to 50 and the site dropped to a CRAWL. I assumed this meant my CPU was maxxed out - but to my surprise, I still had a 50% idle CPU. I am nowhere near my bandwidth limits. So what on earth could be causing the web server to basically freeze?

All I can figure is there is something blocking the httpd processes from finishing. Something that keeps them in the run state instead of the sleep state.

[seindal]

I had a look at the process list above, and 10mb for each apache process, of which 6 shared, is not out of the ordinary. Mine are over 20mb, though that is with mod-perl loaded.

You say you have a lot of pictures, so why don't you run two apaches. Let one serve the dynamic contents (on port 80) and the other the static content (on another common port, say 8080). On you dynamic pages link all images to the other apache on 8080.

For the first server on port 80 you keep your current setup, but the other should be stripped down to an absolute minimum. Adjust the values of MaxClients on the two according the balance of dynamic/static pages you serve.

Another suggestion. Apache works with a master process and a lot of worker child processes. Make sure all your php files are loaded in the master process, so they are shared in memory between all the child processes. I can't quite tell you how to do with php, but there is probably some way to preload the files in the master process. The processes won't be any smaller, but a larger percentage will be shared, reducing the overall demand for memory.

René.

[spagmoid]

Ya I understand that 4 meg unshared is normal, but its not right! Its a crazy amount of RAM for serving a file.

I can't believe Apache could get so popular without having a way to deal with this.. yes maybe I could setup another apache, if the one serving static content could get significantly smaller.

[spagmoid]

Addendum/Correction: It seems the MySql usage was only a part of it, the bigger problem may have been PHP itself. I always had a lot of httpd processes taking up CPU, but since CPU was always 30%+ idle, I assumed that wasn't the problem. I guess maybe PHP/httpd was set not to use all the CPU even if it was the bottleneck - something to do with nice levels? Anyway, after installing Turck MMCache the problem has completely disappeared! Exceeding previous activity levels, and load avg is <0.8, CPU at 80% idle. Yay!