Server Load Question

3 Servers, 100,000 uniques - Too much to ask?

[AZEvil]

I know there are no concrete answers to this, but I'm hoping that someone can at least confirm my thoughts that this isn't as much a hardware issue as a config/code issue:

2 Xeon 3.0 Single CPU servers
1 Xeon 3.0 Dual CPU server
4GB Ram in each server
Apache 2
Running PHP - a modified version of WordPress and a few plugins for WordPress
PHP gd for captchas
Peak hours getting 5,000 - 7,000 uniques - lasting for 8 hours of the day - total of over 100,000 uniques for the day
Servers are behind a Linux Virtual Server load balancer - 50% of traffic to the Dual Xeon - 25% to the other two

Currently these servers are running with an average load of 150+ - is it rediculous for me to think that the servers should be able to handle that kind of traffic?

[jatar_k]

I would think that hardware should be able to handle that level of traffic, though, as you said, there are no concrete answers.

There are many, many more things that would need to be considered though.

Have you tried looking for bottlenecks? starting with the code.

[trillianjedi]

my thoughts that this isn't as much a hardware issue as a config/code issue

Yes, sounds like it.

Database backend? RAM bound, CPU bound, disk bound?

As jatar_k says, you need to identify the bottlenecks, only then can you acertain what you need to do to resolve it.

a modified version of WordPress and a few plugins for WordPress

Probably first port of call.

I'm no expert on configuring multiple servers in this fashion, but I wonder whether you'd be better off with the 2 single CPU boxes dealing with Apache and the dual CPU box running your database, rather than all three doing both tasks?

TJ

[AZEvil]

Actually...I forgot to mention (sorry), because it's not overloaded at all (average load of .37), is there is a 4th server running MySQL only, Dual Xeon, 8GB of RAM, so the three servers in question are specifically web servers only - they are the ones having the problems...

Any suggestions on how/where to pinpoint the bottlenecks?

[jtara]

First, make sure that your machines really ARE over-loaded. The "load" figure is NOT a good way to determine this! It is an almost completely useless for this purpose. It certainly is not a good absolute measure of load.

It can be useful as a RELATIVE measure. It's useful to know that the load on a particular system is twice as much today as it was yeterday, for example.

Load is simply the number of processes in the ready queue (plus the number of processes currently running - i.e. 1 or a single-CPU system, 2 for a dual-CPU system, etc. - assuming at least 1/2 processes are ready to run...)

One system might run happily with a load of 100 and a low CPU utilization. Another system might bog-down with a load of 10 and 100% CPU utilization. It depends on both the speed of the system and the behaviour of processes.

Instead, check average CPU% and I/O utilization, and do some simple practical tests - for example, are web pages slow to load?

The measurements that should matter to you are the measurements that affect your users. Your users don't care about a synthetic number generated by your operating system.

[AZEvil]

Average CPU is above 98% and pages are slow loading.

[trillianjedi]

Average CPU is above 98%

Idle or used?

If used, and if this is only on the apache boxes, that might indicate a php problem - inneficient coding, very tight loops etc.

Is httpd the big CPU hog?

TJ

[AZEvil]

Looks like we found the problem...we trimmed down a couple of the plugins and made some things static...much smoother now. We picked up 3 more servers and now that appears to be overkill. Not much load on anything and looking like we're on track for a total of 150,000+ uniques (and 700,000+ page views) today.

Thanks everyone for the help.