Forum Moderators: coopster
" Removed "template" page generation in favor of inline on-the-fly generation. This reduced amount of memory used per page view by up to 2 megabytes. Not all that important at 10k views a day, but at 100k views, it's critical. "
can anyone explain what the above means? does it mean echoing out data as it is processed instead of storing and spitting out a template?
thanks!
Then the "inline" approach I intepret as embedding the HTML markup and control logic in the "native" site code itself, rather than in a template system. Skipping all the above steps except for the actual execution and output. Possibly echoing it as processing occurs, but not necessarily.
The memory savings is most likely chalked up to skipping the string parsing & execution step. A scripting interpreter (PHP, Perl) is pretty fast and efficient at loading source code and parsing it; implementing an intepreter in an intepreter is not. That's what most template systems do.
For completeness (the subject does read "template theory" :), there is a hybrid approach that combines the above two: pre-compiling a template into executable code. JSP does this (each template page is compiled into a Servlet), some PHP template add-ons do as well I believe.
1) Don't store or load big chunks of HTML code into variables, instead put them directly into the PHP page or echo them directly from a file.
If you need to parse the HTML for something, do it line by line echoing as you go if possible (as opposed to slurping the entire file into an array or variable).
Ideally there will be no need to parse HTML code because you can switch to PHP mode to display variables.
2) If you have to hold a large chunk of data in a variable, do it for the shortest span of time possible.
That will create faster, less memory intensive, scripts. A lot of people will argue, however, that easy of use, extensibility, readability, etc.. are more important.
There's something to be said for all of that, but as you say, at 100k+ daily page views philosophical programming standards start to seem kinda silly :-)
>> That will create faster, less memory intensive, scripts. A lot of people will argue, however, that easy of use, extensibility, readability, etc.. are more important.
i thought as much.
...and because it's a forum, brett can't cache output to disk either - which otherwise might solve some of the ram usage and maintain the usability mentioned above.
many thanks!
Templates are really useful on projects which are big enough that you want your system itself to have security controls for the work that people on the project are doing - e.g., you want your designers to be able to input HTML, and it's not enough to politely ask them to refrain from doing anything except populating some variables.
I'll probably be going down the template route soon enough with one project since that's what I need - even though I rather hate that idea of a whole extra layer of complexity, both for my coding, and for my processor. If php only had some way of natively including files, but specifying in the include statement, 'and don't let the included file do anything except set a few variables' this would already do the trick for me and I wouldn't need to be thinking about this whole extra layer. I suppose I could also use something like
parse_ini_file()but that, again, is straying from the 'native' format. It'd probably be a whole lot faster than a non-caching template system, but I'm guessing then that a lot of designers would be put off with it, whereas if I use smarty templates, they'll think, 'oh, smarty templates, I need to learn those some time,' and I'm counting on designers when it comes to promoting this thing and getting it to catch on.
So if it's just you and a very small group of people you can really trust who are working on the site, and it's likely to stay this way, don't worry about templates; if you later want to allow people you trust less to be able to add or change the HTML styling without needing to check out their work, go templates.
BTW: in July, John Lim updated his article a HOW TO on optimizing PHP [phplens.com]. I've only just come across the new version. He says, amongst other things, that
$var = "this $kind of $doing vars";no longer has any significant speed advantage over
$var = 'this '.$kind.' of '.$doing.' vars';since 4.3.x, though I'll probably stick to the latter for code clarity and old rusty habit. He also points out the surprisingly large speed benefits of using
ob_start()(not even with
ob_gz_handler()) and why just using a buffer is even faster than either spitting out output line by line, or concatenating all content into one long $content variable. This is probably the best PHP speed-optimization article I've come across.
PS: If I've just overlooked something, and there is a way of natively letting an included file include some variables, but not execute any code, please let me know!
john lim's article is one of my favourites on php - i'll have to check out the new version.
i actually use a template system with my site. using eval() and variable substition like {$this}.
the reason i found brett's post so interesting was that i am always looking for ways however to lighten the load, cos we max out our server in our peak season. however as i realised above, brett can't afford to cache any output, we can though - i've just been playing with cache_lite and the scalability improvement is phenomenal.
thanks!
<added>
i've just read some of the changes in the article.
is there a limit on using output buffers during the generation of one page? can one use as many as 10, or even more?
cheers
Regarding variable-substituting templates, they can be sped up with the "compilation" template style- use Perl or whatnot to transform
{$var} to <?= $var?>. But keep the eval() code so that you can test your templates directly without a compilation step during development. The templates can be batch-compiled before you send the code off to production.
that's a really interesting post you refer to - (mostly chinese to me of course ;-)
re: the variable substitution.
i have already been looking at other ways to do this to, and one was for every instance of a template, to use output buffering instead of eval():
ob_start()
include ('html_template.inc');
$data = ob_get_contents();
ob_end_clean();
the templates then become a standard mixture of <?php echo $template_var;?> and html code, like you suggest.
that was the reason i asked about the amount of output buffers per page - could be as many as 8 templates / buffers per page (for formatting the content, sidebars, adds, news articles, etc).
i have not tested the speed differences over eval() yet, but in your opinion does this sound like a more scalable use of templates.
many thanks!
ob_start()
include ('html_template.inc');
$data = ob_get_contents();
ob_end_clean();
This is pretty much exactly what I do, with the same usage pattern you're envisioning for your site. No problems whatsoever. Plus I have some templates that include other templates in the same way, so there are some nested ob_starts() in addition. Still no problems.
thanks very much for the advice.
High memory usage is often related to something completely different - most high traffic sites are usually disk IO bound: php needs data from disk, disk is too slow => more and more processes pile up and consume more and more RAM.
>> most high traffic sites are usually disk IO bound
what does this mean? that the bottleneck on high traffic sites is the disk seek speed?
cheers
Tim
we are lucky enough to just have trusted designers working on the html, so the use of <?php?> tags in included html files works fine. a further layer of security such as provided by smarty is not necessary - i also really balk at the idea of learning the smarty syntax!
something i read regularly is that one shouldn't really bother over-optimising (just like freeflight2 said), but that the real bottleneck on dynamic sites is the database - outpout caching is an order of magnitude quicker and more scalable than any code optimisation.
my favourite analogy (read on some blog somewhere) likened most php code optimisation and decisions such as whether to use foreach() or while() to building a bridge and trying to decide whether to use milwaukee or stanley tools ... lol
I like the idea of having all HTML output for normal pageviews (not including admin or forms) being at least nicely localized in a separate file or directory, so it's all there in one place to be easily changed. This doesn't necessarily mean all the extras of a whole extra templating layer.
Savant looks interesting indeed; this is the kind of thing I do with many of my scripts, which I guess is like having an ad hoc templating system, a sort of informal convention which keeps your HTML in separate pages away from your program's 'business logic.' The kicker is, though, you'll have to look through each of these templates yourself if you don't have a design team you implicitly trust - you won't be able to blindly allow company x to have whoever they want to modify the templates, unless you're willing to deal with the possible consequences - errors due to neglect or purpose, stolen code, etc. A truly separate layer of interpreted template code will do a lot more to help you seal up your ap security-wise if you have to play with other parties whose skill or will is an undetermined quantity. It's time concerns here - does this situation happen often enough that you're willing to deal with the consequences? If it's a script just used by three or four sites, probably not worth going to the hassle of adding that whole extra layer - if it's something you're using on a broader scale, then it's probably worth it.
and using single quotes instead of double quotes for optimization purposes has apparently become one of them with 4.3
I don't see how that's possible. Regardless of how the PHP engine has changed, single quotes still means don't parse for variable substitution.
It might not be a huge speed difference but there has to be a difference.
I agree with the over-optimization, you get to a point where it doesn't really matter UNTIL you get some huge surge of traffic. When that time comes it may be cheaper to throw more hardware at the problem or you can just use a simple caching script(server side or client side or both).
