Apache very very slow suddenly. More than 1 min to respond

[ededoulis]

Hi all!

for the past few days apache has been veeeery slow on my dedicated server. I initially tried everything I could think of, tamprered with apache vars but nothing.

I then migrated to a new dedicated machine with more CPU and RAM (I was going to do that anyway way) but again, same behaviour!

I really need a new perpective on this so any help will be greatly appreciated!

Machine is AMD Opteron 8 cores, 32GB DDR3 and runs CentOS 6 with Plesk

Some more info I forgot: problem seems to be on dynamic pages (PHP+MySQL). Static ones load fast. BUT, if I make a new php and just do not connect to MySQL still same problem. Any thoughts?!?

cheers!

[TypicalSurfer]

Quick way to get a glimpse of what might be going on would be to open a terminal and run top, see if anything jumps out at you while while making your page requests or you can run something like wget locally at your server and watch what resources are being eaten up via top (remote server).

Did you migrate your old my.conf (mysql configuration file) from your previous server? If so, I'd start there.

[ededoulis]

Below is the output of top. As I mentioned I have tried anything that could come to mind during the past 3-4 days, hence the fresh approach needed :)

No, I did not migrate anything other than the website (files + db)

top - 20:17:07 up 2:32, 1 user, load average: 0.49, 0.44, 0.43
Mem: 32814320k total, 1967780k used, 30846540k free, 46720k buffers
Swap: 2097136k total, 0k used, 2097136k free, 330620k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1895 root 20 0 523m 3232 964 S 0.0 0.0 16:32.89 sw-collectd
1525 mysql 20 0 4243m 107m 6368 S 0.0 0.3 1:51.41 mysqld
1870 drweb 20 0 160m 156m 1700 S 0.0 0.5 1:02.16 drwebd.real
39 root 20 0 0 0 0 S 0.0 0.0 0:04.53 events/4
1885 root 20 0 320m 23m 4380 S 0.0 0.1 0:03.25 sw-engine
1626 root 20 0 143m 52m 3108 S 0.0 0.2 0:02.67 spamd
392 root 20 0 0 0 0 S 0.0 0.0 0:02.63 md1_raid1
2081 apache 20 0 332m 17m 5288 S 0.0 0.1 0:01.71 httpd
1000 root 20 0 0 0 0 S 0.0 0.0 0:01.69 flush-9:1
1782 apache 20 0 331m 16m 5272 S 0.0 0.1 0:01.11 httpd
1793 apache 20 0 331m 16m 5492 S 0.0 0.1 0:00.96 httpd
1966 apache 20 0 331m 16m 5488 S 0.0 0.1 0:00.96 httpd
1740 root 20 0 327m 19m 10m S 0.0 0.1 0:00.92 httpd
2076 apache 20 0 331m 16m 5268 S 0.0 0.1 0:00.87 httpd
1819 apache 20 0 333m 18m 5540 S 0.0 0.1 0:00.85 httpd
2129 apache 20 0 331m 16m 5268 S 0.0 0.1 0:00.83 httpd
4055 apache 20 0 331m 16m 5444 S 0.0 0.1 0:00.80 httpd
4445 apache 20 0 333m 17m 5252 S 0.0 0.1 0:00.80 httpd
2542 apache 20 0 333m 18m 5852 S 0.0 0.1 0:00.79 httpd
4427 apache 20 0 330m 15m 5220 S 0.0 0.0 0:00.79 httpd
4403 apache 20 0 331m 17m 5472 S 0.0 0.1 0:00.77 httpd
1372 root 20 0 51604 2424 1828 S 0.0 0.0 0:00.75 master
4729 apache 20 0 331m 16m 5480 S 0.0 0.1 0:00.75 httpd
2503 apache 20 0 333m 17m 5252 S 0.0 0.1 0:00.72 httpd
1988 apache 20 0 333m 17m 5260 S 0.0 0.1 0:00.70 httpd
2263 apache 20 0 332m 16m 5256 S 0.0 0.1 0:00.70 httpd
2671 apache 20 0 332m 16m 5452 S 0.0 0.1 0:00.68 httpd
2079 apache 20 0 331m 16m 5508 S 0.0 0.1 0:00.67 httpd
1965 apache 20 0 333m 17m 5280 S 0.0 0.1 0:00.66 httpd
2695 apache 20 0 332m 16m 5280 S 0.0 0.1 0:00.66 httpd
2933 apache 20 0 331m 16m 5236 S 0.0 0.1 0:00.66 httpd
2272 apache 20 0 331m 16m 5476 S 0.0 0.1 0:00.65 httpd
2689 apache 20 0 331m 16m 5208 S 0.0 0.1 0:00.65 httpd
4726 apache 20 0 333m 17m 5208 S 0.0 0.1 0:00.64 httpd
3856 apache 20 0 331m 15m 5276 S 0.0 0.0 0:00.63 httpd
2121 apache 20 0 331m 16m 5268 S 0.0 0.1 0:00.62 httpd
4377 apache 20 0 331m 16m 5280 S 0.0 0.1 0:00.62 httpd
2880 apache 20 0 333m 17m 5272 S 0.0 0.1 0:00.61 httpd

Server is using a fraction of available resources

So as to get a feel of the problem, the site served though the server is www.scoreline.com.cy

An example of a static page that does not seem to have any issues is at [scoreline.com.cy...] (I just saved the home page in html and uploaded it).

Thanx for your help

[TypicalSurfer]

If you are running drupal/wp, I'd consider going through and selectively disabling modules or plugins, it looks like something is hitting MaxClients ceiling causing new requests to hang.

Also look into MaxClients server setting, with all that RAM you could probably bump it unless you've already tuned for that but I would lean toward flaky plugin causing bad behavior.

[ededoulis]

It's not Drupal/WP, it's a custom made cms. The thing is I have used the same cms on dozens of sites and even this one was working perfect until a few days ago.

As for MaxClient, I tried raising to even 1000 but no change, and even after a server reboot when you would expect traffic to be still minimal the behavior is still the same.