Load average is high for no reason

[enchant]

No reason that I can comprehend, anyways...

 4:33pm up 16 min, 2 users, load average: 5.99, 3.01, 1.59 
76 processes: 75 sleeping, 1 running, 0 zombie, 0 stopped 
CPU states: 7.5% user, 1.5% system, 0.0% nice, 90.8% idle 
Mem: 517672K av, 370880K used, 146792K free, 113688K shrd, 196644K buff 
Swap: 705424K av,  0K used, 705424K free     58312K cached 

 PID USER  PRI NI SIZE RSS SHARE STAT LIB %CPU %MEM TIME COMMAND 
 1496 webdemon 11 0 6156 6156 868 D  0 4.5 1.1 0:00 ultimatebb.cgi 
 347 pat  1 0 944 944 752 R  0 0.3 0.1 0:02 top 
 7 root  -17 -20  0 0  0 SW<  0 0.1 0.0 0:03 raid5d 
 1487 root  5 0 1280 1280 808 D  0 0.1 0.2 0:00 sendmail 
 1495 root  5 0 1264 1264 824 D  0 0.1 0.2 0:00 sendmail 
 1 root  0 0 188 188 160 S  0 0.0 0.0 0:05 init 
 2 root  0 0  0 0  0 SW  0 0.0 0.0 0:00 kflushd 
 3 root  0 0  0 0  0 DW  0 0.0 0.0 0:00 kupdate 
 4 root  0 0  0 0  0 SW  0 0.0 0.0 0:00 kswapd 

Over the past couple of weeks, my load average has been regularly skyrocketing. It used to stay below 0.5, but now it's often up around 5, and occasionally up near 50!

I can't see any processes that are individually causing problems. 90% cpu idle time. No memory swapping. What else could cause this problem? Slow disks? I'm using a RAID5 system with three IDE drives.

If anyone can see anything in the top output above or can give me suggestions of what else I could look at, I'd be very much appreciative.

[SeanW]

Sounds like web traffic to your cgi. Especially in a forum, where you'll have a lot of DB IO, you'll see high one minute load averages. You just have a lot of processes on the run queue, probably waiting for IO. If your performance is acceptable, then a high load average is ok.

You can also watch iostat to see if you're spending a lot of time on the disk...

Is your .cgi in perl? Running it under mod_perl may help.

Sean

[martin]

If you can't do mod_perl try SpeedyCGI it is more forgiving to sloppy code than mod_perl.

PS. Most php apps are suited to run as mod_php which is faster than perl cgi, I'd try to use php for new apps if possible.

[SeanW]

which is faster than perl cgi, I'd try to use php for new apps if possible

Not trying to get into a religious war, but blanket statements like that are generally false.

Some of the largest sites on the net run under perl.

Each language can be misused and give poor performance. One only has to look at software like Nuke and some of the wikis out there to see how PHP can be abused into giving an unmaintainable and slow application. Same goes for Perl, Python, and whatever language you want to throw out there.

I've been using PHP since 2.0 and Perl for even longer, and I know it's not as simple as one language over another. Each language does things differently, and is better for particular situations.

Sean

[martin]

>Not trying to get into a religious war, but blanket statements like that are generally false.

The technology is faster, which means that most of the apps will be faster too. I'm not comparing mod_php vs mod_perl, in this case perl obviously wins. Lots of the perl code I've seen though won't run as mod_perl.

>Some of the largest sites on the net run under perl.

That doesn't mean anything. Perl is the oldest high level language for dynamic sites and this is probably the greatest influence, not speed.

>Each language can be misused and give poor performance.

That's why in a perfect world you'd want something that does exactly what you want - nothing more or nothing less. Pretty much impossible unless the app is developed for you which would cost more than better hardware.