homepage Welcome to WebmasterWorld Guest from 54.205.119.163
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Code, Content, and Presentation / Apache Web Server
Forum Library, Charter, Moderators: Ocean10000 & incrediBILL & phranque

Apache Web Server Forum

    
Holding F5 dramatically slows down web server
nosterus




msg:4187308
 2:40 pm on Aug 15, 2010 (gmt 0)

Hello,
I'm trying to find out why holding F5 key (instantly refreshing page) slows down web server. How to avoid this?

I noticed this symptom on many web pages not only on mine.
Sometimes it happens that all other other pages on the same server(ip) stop responding. After few minutes it all returns to normal state and everything works fine again.

For testing I installed freeBSD with Apache 2.2 and php5.3.2 and mysql5.1x only to analyze this behaviour.

On this video you can see 'top' command and how number of proccesses rise when I hold F5 only on one page.
video: [wsninja.com...]

I also found out that this happens only when refreshing dynamic pages. (on static files this doesnt happen)
Probably it has something to do with mysql connections but I run 5 queries with execution time less than 0.01s / per page load.

Many sites(not all) I tested have this issue.
Do you have any ideas how to fix this?

Thanks for help

 

nosterus




msg:4187325
 3:33 pm on Aug 15, 2010 (gmt 0)

Here is what happens while holding F5 on default wordpress3.0.1 configuration. It eats all ram and swap...

Has anybody made some tests like this one?
Install wordpress3.0.1 (it takes less than 2 minutes)
Then hold the F5 key on the first(Welcome to WordPress) page.
Check the server resources rising.

[wsninja.com...]

It has to be some server configuration trick
Please help

jdMorgan




msg:4187333
 4:04 pm on Aug 15, 2010 (gmt 0)

This is a very "big" subject area.

Look into cacheing. Look into KeepAlive, KeepAliveTimeout, MaxKeepAliveRequests, the various RLimit directives, and all of the other server-module config settings that control how HTTP requests relate to file, script, and database requests.

Be aware that each HTTP request stands on its own. The server itself has no memory of such things as "How many previous requests have been received (per unit time) from this IP address?" Tracking that kind of thing requires a script based on IP addresses, or in the case where the IP address can change due to caching-proxy use (e.g AOL and EarthLink ISPs), you'd have to track by using a session cookie (for clients that will return a cookie - not all will). Of course, this extra work only adds to the server load...

One option --should all others fail-- is to use a script to track how many database connections are currently open (this script must start with the server and run continuously), and then return a 503-Service Unavailable along with a Retry-After HTTP response header whenever that number exceeds some tested limit.

Jim

nosterus




msg:4187343
 4:43 pm on Aug 15, 2010 (gmt 0)

Thanks for answer jdMorgan.
I agree that cacheing would help a lot. But not all sites can be served 100% staticaly. (user profiles and so on...)

"script based on IP addresses"
Do you mean counting how many requests were made from the same user (ip/cookie) in some period? If yes where would you save that data? Database or file?

If user make more requests than allowed he should recieve "Retry-After HTTP response header". Correct?

Thats actually a pretty good idea.

And my apache settings for info:
KeepAlive On
MaxKeepAliveRequests 300
KeepAliveTimeout 2

AlexK




msg:4187344
 4:46 pm on Aug 15, 2010 (gmt 0)

nosterus:
this happens only when refreshing dynamic pages

That is the reason why.

More complete explanation:
When any server sends a static-html page, it auto-sends a Last-Modified header, possibly with a Cache-Control header that explicitly allows the page to be cached for a fixed period of time. When you press `f5' in your client, the client will either immediately refresh from local-cache or, at the very most, send a request that includes a If-Modified-Since request header, which will result in a `304 Not Modified' response (tiny, virtually no resources).

Dynamic pages? No such headers are sent - it is entirely your responsibility to send them.

Thus, every `f5' causes the entire script to have to be re-run. If your keyboard is polling at 30 times / sec, that is how often the script will be called.

(The above static-file behaviour is called `Content Negotiation'. I've produced a Class to handle it for PHP scripts, called `Conteg'. On my site to download.)

nosterus




msg:4187358
 5:35 pm on Aug 15, 2010 (gmt 0)

AlexK: I'm just reading your Conteg intro and it is very interesting. I will take a closer look for sure. Will post here if/when I fix my problem.

Thanks a lot

nosterus




msg:4187380
 7:00 pm on Aug 15, 2010 (gmt 0)

AlexK:

Here is example how I'm testing Conteg.

<?php
ob_start();
include('classes/class.Conteg.php');

session_start();
include_once('./config.php');
generatePage();

new Conteg();
?>


Headers then look like this:
RESPONSE
DateSun, 15 Aug 2010 19:39:32 GMT
ServerApache/2.2.15 (FreeBSD) mod_ssl/2.2.15 OpenSSL/0.9.8k DAV/2 PHP/5.3.2 with Suhosin-Patch
X-Powered-ByPHP/5.3.2
Last-ModifiedSun, 15 Aug 2010 19:39:32 GMT
ExpiresSun, 15 Aug 2010 20:39:32 GMT
Content-Languageen
VaryAccept-Encoding,User-Agent
Cache-Controlno-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragmano-cache
Status200 OK
Content-Encodinggzip
Content-Length1599
X-Content-Encoded-Byclass.Conteg.0.13.8
Keep-Alivetimeout=5, max=100
ConnectionKeep-Alive
Content-Typetext/html; charset=ISO-8859-1

REQUEST
Host domain.com
User-AgentMozilla/5.0 (Windows; U; Windows NT 6.1; en-GB; rv:1.9.2.8) Gecko/20100722 Firefox/3.6.8
Accepttext/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Languageen-gb,en;q=0.5
Accept-Encodinggzip,deflate
Accept-CharsetISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive115
Connectionkeep-alive
CookiePHPSESSID=ac2fbdcaqubbkfq0510o5hurr5
Cache-Controlmax-age=0

And it doesn't help. Still the same problem. Any other ideas? Are you sure that should solve my problem?
Thanks in advance

AlexK




msg:4187451
 12:05 am on Aug 16, 2010 (gmt 0)

I'm a bit worried by the lack of colons (`:') & spaces! I assume that that is your copy/paste.

It is working fine, and doing what you have asked. You have *not* included something that you may need (see later). Here is a quick breakdown of what you have pasted:
Last-Modified: Sun, 15 Aug 2010 19:39:32 GMT
Expires: Sun, 15 Aug 2010 20:39:32 GMT

Those are Conteg defaults: Last-Modified at current-date & Expires at 1 hour. Up to this point, the browser will refresh from local-cache for an hour, then send a Last-Modified.

Content-Encoding: gzip

Good. That's up to an 80% reduction in page size, and consequent faster delivery.

Cache-Control: no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache

Conteg non-default (default is to `cache-all'). Invalidates the earlier instructions. Whoops.

You need to investigate
session.cache_limiter (defaults to nocache).

Conteg does take account of PHP sessions, but only for setting eTags (the default for which is `none') (eTags are a method for getting some cache-value on dynamic pages - read the info in Conteg).

Your use of PHP-Sessions probably invalidates this next point, but if you want to get full value, you need to institute some form of page-dating. By default, the `modified' Conteg setup parameter is set to
time(), which is the current date-time. If your pages are truly dynamic (capable of changing on every access) that's what you want. If your pages contain fixed-info (which may be updated later) then you need to store a last-modified date & pass that to Conteg. However, that would also be negated by the current Sessions setup.
nosterus




msg:4187890
 7:10 pm on Aug 16, 2010 (gmt 0)

Hello,
first of all I really appreciate your help Alex.

I'm a bit worried by the lack of colons (`:') & spaces! I assume that that is your copy/paste.
Yes this is my copy/paste bug.

Now I managed to make it work - somehow. I will explain later why i said 'somehow' :)

The code:

<?php
ob_start();
session_cache_limiter ("private_no_expire, must-revalidate");
session_start();

include('classes/class.Conteg.php');
$param = array(
'use_etag' => true,
'expiry' => 84600
);
$_Instance = new Conteg( $param );

include_once('./config.php');

//here all the logic and database queries happen
//and the page output
generatePage();

$param = array(
'modified' => 1281909504
);
$_Instance->setup( $param );
$_Instance->show();
?>


I guess headers are now ok:

DateSun, 15 Aug 2010 22:40:59 GMT
ServerApache/2.2.15 (FreeBSD) mod_ssl/2.2.15 OpenSSL/0.9.8k DAV/2 PHP/5.3.2 with Suhosin-Patch
X-Powered-ByPHP/5.3.2
Last-ModifiedSun, 15 Aug 2010 21:58:24 GMT
ExpiresMon, 16 Aug 2010 22:10:59 GMT
Content-Languageen
VaryAccept-Encoding,User-Agent
EtagW/"5c4658281acbe6e136574a9aac5a168f"
Status200 OK
Content-Encodinggzip
Content-Length834
X-Content-Encoded-Byclass.Conteg.0.13.8
Content-Typetext/html; charset=ISO-8859-1


Some extra info:

Last ModifiedMon Aug 16 2010 20:17:34 GMT+0200 (Central Europe Daylight Time)
Last FetchedMon Aug 16 2010 20:17:34 GMT+0200 (Central Europe Daylight Time)
ExpiresTue Aug 17 2010 00:10:59 GMT+0200 (Central Europe Daylight Time)


But in this case the script (generatePage()) with all the queries is still executed every time. Before $conteg->show() method is called ? How can I avoid this?

I can achieve this only manually by moving the code after the first load (when cache is saved) higher:
if($_Instance->show()){
exit;
}
generatePage();

before generatePage()

In that case page loads with lightspeed without running any queries or logic from the "generatePage()"

My idea is to save last modification time (when user commented something or new content was added, ....) in temporary cache file. And then on every page load I would init Conteg with 'modified' parameter from cached file.

I think now I managed to reduce bandwidth but scripts are still taking unnecessary server load.

AlexK




msg:4188086
 3:42 am on Aug 17, 2010 (gmt 0)

Hi nosterus.

We should really be carrying out this conversation on my forums but--since Conteg was originally promoted via WebmasterWorld--that's fine.

Health warning beforehand:
I'm unsure on the precise Response Headers used by Browsers & those used by Proxy Servers. I doubt that anyone in the world actually knows for sure, since all browsers differ, and Proxy Servers (in my experience) are a law unto themselves, often flagrantly ignoring relevant RFCs. So, take certain confident statements below with a pinch of salt.

As Conteg stands in the latest version (yours - v0.13.8) a 304 Not Modified can only be sent during
Conteg::show(). Clearly, that needs fixing, since it means that the whole page has to be produced before the server decides not to send any of that resource-intensive work. I'll get onto it straight away (an extra function would seem to be the obvious method - if you have some good ideas, post now).

Other Response headers:
The Response Headers you quoted are missing all Cache-Control & Pragma headers. Conteg default is 'cache-all', so that must be Session post-processing, although I have also experienced server post-processing with Vary headers, so examine that option also.

As I understand them, after exhaustive readings of the various RFCs, Cache-Control headers (and Pragma: (no)cache before them) are designed principally for Proxies & not browsers. However, I reckon that most browsers make use of them.

Many ISPs put Proxies between the browser & the internet. For the ISP, the Proxy is an opportunity to keep all responses within their own network, which means an opportunity to save money. In pursuit of that, I've had examples from my own customers of situations where the Proxy completely ignored explicit browser instruction to return a fresh page, and returned a 304 instead (the exact opposite problem to your OP) (the fix is to either use a POST instead of GET, or to switch to HTTPS).

I'm unsure of the interaction on browsers between Not Modified & eTag. The former is http 1.0 & the latter 1.1. The eTag with `W/' prefix is clearly designed for dynamic pages, but I've never had the courage to abandon a Last-Modified header.

I'll get onto a method to produce a 304 without having to generate an entire page - clearly needed (so thanks for working with us so far).

nosterus




msg:4188763
 6:25 pm on Aug 18, 2010 (gmt 0)

Hi Alex, I'm very happy to hear that you're going to implement this new method.

Initializing the Conteg class with the last-modified-date parameter and sending 304 immediatly without need to generate the whole page will be a great function. It can really save lots of servers resources. ( save energy, save the planet :D )

If I can help you somehow please let me know. (maybe for testing)

nosterus




msg:4188800
 7:45 pm on Aug 18, 2010 (gmt 0)

I made small modification and it now works the way I wanted.

<?php

session_cache_limiter ("cache-all");
session_start();

include('classes/class.Conteg.php');

$page_has_changed_at = 1281941681; //get timestamp of the last change from DB or file
$_Instance = new Conteg( array('noprint' => true));
$_Instance->show_and_die($page_has_changed_at);

include_once('./config.php');

//here all the logic and database queries happen
//and the page output
generatePage();

$param = array(
'modified' => $page_has_changed_at
);
$_Instance->setup( $param );
$_Instance->show();

?>

And here is the code of the new show_and_die() method:

function show_and_die($page_has_changed_at){
if(isset($_SERVER['HTTP_IF_MODIFIED_SINCE'])){
if(strtotime($_SERVER['HTTP_IF_MODIFIED_SINCE']) >= $page_has_changed_at){
$this->setup(array('modified'=>$page_has_changed_at));
$this->show();
exit;
}
}
}


That works for me tested on Apache 2.2

Cheers

AlexK




msg:4189578
 3:38 am on Aug 20, 2010 (gmt 0)

nosterus:
I made small modification

Yes. I wish that my mods could be small... took me 24 hours just to get my head back up inside the entire HTTP/1.0 & HTTP:/1.1 specs (I used the time usefully, improving the `Content Negotiation' section within `Help & Advice'). Nevertheless, all changes are made & ready to be bug-checked. The main issue boiled down to ensuring that
_initResponse() could be multi-entry (originally designed to be called just once).

From the outside, the Class works identically as before, but now contains an extra function:
_isResponseNoContent() which returns either (304 or 406 or 412) or FALSE. If the former, then show() & exit(). If the latter, carry on as normal.

If I can help you somehow please let me know. (maybe for testing)

That would be excellent, particularly because I do not use SESSIONs. If you will sticky me with your email address, I shall post the new version to you once bug-checked at my end.

alias




msg:4189884
 4:33 pm on Aug 20, 2010 (gmt 0)

A little input from me -- to protect the server from such hammering, take a look at apache's mod_evasive.

nosterus




msg:4189938
 6:10 pm on Aug 20, 2010 (gmt 0)

Yes alias, thats true. Also mod_limitipconn can help in such cases. Probably all together is the right combination.

AlexK




msg:4191451
 3:28 pm on Aug 24, 2010 (gmt 0)

No further response from nosterus, although I did mail him the latest version.

Conteg v0.13.9 has been available for download for a few days now, and tests out fine on my site:
Download [download.modem-help.co.uk]

nosterus




msg:4192444
 10:37 am on Aug 26, 2010 (gmt 0)

I'm on vacation and it's pretty hard to find wireless here.
Right now I'm ocupated with testing beer but I will test new conteg and report it here when I come back for sure.

cheers

nosterus




msg:4193948
 9:56 am on Aug 30, 2010 (gmt 0)

I've tested it using sessions in different browsers (firefox3.6, ie8, chrome5,..) and it works like a charm.

With isResponseNoContent() in now possible to catch 304, 406 or 412 before server builds the whole page.

It is a great feature, thanks Alex!

AlexK




msg:4194704
 6:59 pm on Aug 31, 2010 (gmt 0)

nosterus:
It is a great feature, thanks Alex!

You are welcome.

It should have been there since the beginning, so thanks for raising the issue.

nosterus




msg:4201586
 11:45 am on Sep 14, 2010 (gmt 0)

Hi, one more thing.
Every time I refresh page with F5, server returns 403 not modified or 200 if 'modified' has newer time() than browser cache. That's ok.

But if I don't refresh page and I only click from homepage to specific subpage and then back on homepage, even if in the meantime 'modified' time has changed, I don't see changes. Unless I refresh page.
For example I have a Poll and every time user votes I change 'modified' time. I don't see changes until I refresh page.

It looks like that 'modified' parameter doesn't compare with browser's last modified time unless I refresh page with F5. Only clicking on subpages from the mainmenu does nothing.

Do you have any ideas why is this happening?

$modified = file_get_contents('modified_timestamp');
$_Instance = new Conteg();

$param = array(
'modified' => $modified,
'charset' => 'UTF-8',
'type' => 'text/html',
'use_content_lang' => TRUE,
'use_content_type' => TRUE,
'cpu_number' => 8,
'use_etag' => TRUE,
'use_accept_charset' => FALSE
);
$_Instance->setup( $param );
if( $_Instance->isResponseNoContent()) {
$_Instance->show();
exit();
}

generatePage();

$config = array(
'modified'=>$modified,
'charset' => 'UTF-8',
'type' => 'text/html',
'use_content_lang' => TRUE,
'use_content_type' => TRUE,
'cpu_number' => 8,
'use_etag' => TRUE,
'use_accept_charset' => FALSE
);
$_Instance->setup($config);
$_Instance->show();

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Code, Content, and Presentation / Apache Web Server
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved