scalable PHP

[httpwebwitch]

When upscaling an app to handle heavier traffic, I figure that the lowest-hanging fruit are, in rough order:

- SQL optimization (indexes, joins, etc)
- output caching
- hardware (more RAM)
- code optimization (optimizing regex, include, and other expensive ingredients)
- data partitioning (and denormalizing)
- load balancing
- database replication (masters & slaves)
- asynchronicity

it's a naive list... Without some profiling, you don't know where the bottlenecks are. And data partitioning could be simple if you have a nice modular DAL, otherwise it could require a major refactoring of your app. I suppose you could just work down the list until you hit the clog, but that's a lot of wasted time.

Regardless, having this list feels good. It's like a shopping list of possibilities.

Is anything missing from it?

[mack]

I would add..

- Bandwidth
- Optimize your output markup (we think in terms of code but we're still spitting out html)
- Optimization of the mysql config (my.cnf) file if its a dedicated mysql server


Mack.

[httpwebwitch]

It's hard to keep "scalable" separate from "fast"... It is possible to have an app that's slow, but scales well and is consistently slow and never crashes under heavy loads. A lot of techniques will straddle that line, and a solution that's slow on the back-end usually isn't scalable.

I just remembered:

- dedicated server or subdomain for serving images & other external assets

[topr8]

ruthlessly blocking scrapers and unwanted bots/spiders as early as possible ... can save a lot of wasted processing.

[brotherhood of LAN]

- output caching

Expanding on that:

- Use If-Modified-Since headers
- Use Last Modified for scripts/images/static pages that do not require re-fetching at all, or very rarely.

[httpwebwitch]

>> ruthlessly blocking scrapers

ooh, good one. I've seen entire networks go bellyup under the weight of an aggressive Slurp crawl. Had to do some equally agressive throttling by useragent.