Forum Moderators: phranque
I run a large retail website, and the content (images, mainly) for the majority of our products are provided by the distributor, who operates a privately accessible content server. Currently, we have a simple mod_rewrite set up that directs these image requests to to a php script, which then asks for the image from the content server, and if it is available, serves it to the user. If it is unavailable, the php script serves our 'Image Not Available' image instead.
The problem we are running into with this approach is that it is putting an enormous load on our server when several images are being requested per second. Since each image request starts a new php process, and each process has to wait for the response from the content server before it can close, the processes start building up quickly.
Can someone provide an example of how to handle this with mod_rewrite? Our main objective is to lessen the load on the server for these image requests, so if it is simply less efficient to do this with mod_rewrite then it would be with php (the current setup), that advice would be valuable as well.
Thanks!
The problem is that you will lose the "file exists" check that you describe. It will be entirely up to the vendor's server to provide an alternate image if the requested image does not exist. This is the downside of proxying and not using a method that involves "waiting for the response" as you described.
You might consider modifying your script to actually copy the image to your server. Then, when an image is requested, you check the local cache (really just an alternate directory path) first. If present, serve the local copy. If not present check the vendor site (the way you do now). If the image is present on the vendor site, copy it to the local cache and serve it. Otherwise, serve the "missing image" file. Once a day, at low-traffic time, you can flush your local copies to prevent serving stale/discontinued images.
Look into general discussions of script-based caching and compression; A very common subject is keeping cached and/or compressed copies of dynamically-generated pages, and the techniques are very very similar. The basic idea is to locally cache static copies of content that is otherwise slow to serve.
Jim
My main goal here is to reduce server load. I am curious, would it help to use the proxy function of mod_rewrite to get the image to our server, then let our php script do it's decision making? That way the script process could get done without the wait and not build up all of these httpd processes.
Would this be more efficient then the way we are doing it now, and allow us the same functionality?
The reduction in "load" provided by local caching is a reduction of bandwidth and CPU load, at the expense of some additional disk I/O (which is still a lot faster than passing an HTTP request). The other alternative, if you need a major load reduction, is to simply set up your own image server on a subdomain (like yimg.yahoo.com), upload a copy of all needed images to it on a periodic basis, and link to those images directly instead of using a script to proxy them as you are doing now.
Jim
