| 10:38 am on Jul 18, 2012 (gmt 0)|
memory issues, maybe? have a look and see how much is being used
| 4:39 pm on Jul 18, 2012 (gmt 0)|
Memory is fine. I have also rebooted the computer a few times. I have many GB's of RAM and it is a heavy multi-core system. The Apache process is barely using any CPU at all like in other posts I have seen with the issue.
My other server services are working fine (remote desktop, ftp, filesharing, media streaming, etc). Just Apache is not serving web pages in anything close to a timely manner.
| 4:51 pm on Jul 18, 2012 (gmt 0)|
This is showing a 250KB page taking 18 minutes to serve... The actual file transfers are fast. This is from the server itself (so local). But the waits are forever.
Image: [s11.postimage.org ]
| 5:46 pm on Jul 18, 2012 (gmt 0)|
the other thing i would think of is DNS issues
| 6:35 pm on Jul 18, 2012 (gmt 0)|
The DNS's are good too. The DNS lookup part of the requests take less then a millisecond.
I believe I am actually "under attack" but not sure what to do about it yet. Looking at the active TCP connections I see many hundreds of connections. Generally there are 5 to 10 from the same IP requesting port N, N+1, N+2, N+3, N+4, etc. Then after those timeout, the same IP is back requesting N+5, N+6, N+7, N+8, N+9, etc. This is coming from many IPs, not just one.
I think because it is so many IPs and they are scanning slowly but in numbers, my Intrusion Detection Policy on my Router is not picking it up... I have never had to deal with it before.
| 7:12 pm on Jul 18, 2012 (gmt 0)|
what about your database processes. maybe they are taking ages for some reason, and eventually timing out. one IP could generate a load of those on every page, depending on how you've written your scripts.
if you've got your databases on another server then a reboot might not have fixed anything
| 7:24 pm on Jul 18, 2012 (gmt 0)|
Just throwing this out there because I had similar symptoms one time without any errors showing up - check to see if there is an excessively large log file somewhere. I had a log file that wasn't rotating properly and exceeded the server file size limit. The pages on the server would load but it would take forever.
| 2:31 am on Jul 19, 2012 (gmt 0)|
you should whitelist only the ports you need for your applications at the firewall.
thus these connection attempts will fail and your server will never see those requests.
| 4:50 pm on Jul 19, 2012 (gmt 0)|
Unfortunately I have services that need many ports open, ideally all ports. So this will continue to happen.
I would be less concerned and try to block ports at the server if apache was eating all the resources, but it is not. It is using 207 threads and 0.1% average CPU load with 150MB of RAM (out of 16GB reserved for web-stuff). So there is a bottleneck somewhere but I don't know where.