Welcome to WebmasterWorld Guest from 35.172.100.232

Forum Moderators: coopster & jatar k

Message Too Old, No Replies

More PHP image upload problems

     
2:39 pm on Mar 19, 2007 (gmt 0)

Full Member

10+ Year Member

joined:June 24, 2004
posts:202
votes: 0


Finally figured out what was causing some image uploads issues when php/gd would attempt to resize the uploaded file (jpeg).

It wasn't the size of the image (mb or kb) but rather the dimensions of the picture. For example, an image which was 300kb and 2500 x 2500 pixel dimensions PHP or GD cannot rescale without bytes exhausted error, but a 2mb 1200 x 1200 pixel dimension resized no problem.

How can I fix this issue? Is this a GD limitation or PHP 5.2?

2:52 pm on Mar 19, 2007 (gmt 0)

Administrator

WebmasterWorld Administrator coopster is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:July 31, 2003
posts:12555
votes: 3


Not sure. What is your memory_limit [php.net] set at?
What function is giving the error message?
Can you offer the exact message syntax (less specifics of course)?
4:11 pm on Mar 19, 2007 (gmt 0)

Full Member

10+ Year Member

joined:June 24, 2004
posts: 202
votes: 0


Memory Limit is set to 33mb.

The two functions I am using to resize photos are:
imagecopyresampled() THEN imagejpeg()

5:46 pm on Mar 19, 2007 (gmt 0)

Full Member

10+ Year Member

joined:June 24, 2004
posts: 202
votes: 0


I think I narrowed the problem down to this function call: imagecreatefromjpeg()

It appears that if the dimensions are too large PHP or GD chokes... Anyone have suggestions?

6:27 pm on Mar 19, 2007 (gmt 0)

Administrator

WebmasterWorld Administrator coopster is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:July 31, 2003
posts:12555
votes: 3


I can create a new jpeg of 3500 x 3500 no problem. Then, I can create another image from this jpeg without issue. You'll have to turn up your error_reporting and see the exact message you are receiving.
6:35 pm on Mar 19, 2007 (gmt 0)

Preferred Member

10+ Year Member

joined:July 31, 2006
posts:629
votes: 0


It could be a GD library problem. Please let us know exact GD version. For example mine is "bundled (2.0.28 compatible)".

use phpinfo() function to get the info.

6:40 pm on Mar 19, 2007 (gmt 0)

Full Member

10+ Year Member

joined:June 24, 2004
posts: 202
votes: 0


I am seeing sporadic reports on some searching for this issue (dimensions - gd - imagecreatefromjpeg). Nothing however which tells of fixing the issue.

GD INFO:
GD Version - bundled (2.0.28 compatible)
FreeType Support - enabled
FreeType Linkage - with freetype
FreeType Version - 2.1.9
GIF Read Support - enabled
GIF Create Support - enabled
JPG Support - enabled
PNG Support - enabled
WBMP Support - enabled
XBM Support - enabled

7:07 pm on Mar 19, 2007 (gmt 0)

Administrator

WebmasterWorld Administrator coopster is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:July 31, 2003
posts:12555
votes: 3


Can you please provide the exact error message syntax (less specifics)?
7:14 pm on Mar 19, 2007 (gmt 0)

Full Member

10+ Year Member

joined:June 24, 2004
posts: 202
votes: 0


Error Message: PHP Fatal error: Allowed memory size of 35651584 bytes exhausted (tried to allocate 10200 bytes) in /path/to/script on line #*$!

Two things, First, I know I can up the limits in the script itself. Is their any danger in doing this? Slow down the site, all other scripts become paused, etc.?

Secondly, been trying some test jpegs, a 2500x3300 jpeg will cause the error, but 2500x2500, no problems. Going to try a few other and see if I can pin it down to an exact size...

7:24 pm on Mar 19, 2007 (gmt 0)

Administrator

WebmasterWorld Administrator coopster is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:July 31, 2003
posts:12555
votes: 3


Allowed memory size of 35651584

35651584/1024 = 34816/1024 = 34MB

You are hitting your memory limit.

7:26 pm on Mar 19, 2007 (gmt 0)

Preferred Member

10+ Year Member

joined:July 31, 2006
posts:629
votes: 0


FYI: You could try setting memory_limit to -1

Note that to have no memory limit, set this directive to -1.

Note: by setting it to -1 some poorly written scripts can eat up server memory. So think it would be better to set it to some "acceptably high" number

7:35 pm on Mar 19, 2007 (gmt 0)

Preferred Member

10+ Year Member

joined:July 31, 2006
posts:629
votes: 0


Is their any danger in doing this?

Less simultaneously run php scripts (because of more memory for each script).
3:48 pm on Mar 22, 2007 (gmt 0)

Full Member

10+ Year Member

joined:June 24, 2004
posts:202
votes: 0


Finally the conclusion:

Added more memory to the server and increased the php memory limits for this one script to parse the large files. This solved the problem but I still have one nagging question.

The image that was causing the limit error was 240kb and 3000 x 3000 pixels in dimensions. An image with smaller dimensions 2400 x 1200 and 1.6mb runs fine (on the old memory limits).

This concludes to me that image dimensions matter more than image size. Given the fact that more and more people are taking larger and larger pictures (picture dimensions), what are the alternatives to GD? Can PHP pass an uploaded file to an alternative image processor to compress a large photo, say through exec()?

My current limits sit at 50mb, with 2gb of system memory this could be eaten up fairly quickly if 20 users decide to upload large files bringing the system to a screeching slow down....

4:17 pm on Mar 22, 2007 (gmt 0)

Administrator

WebmasterWorld Administrator coopster is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:July 31, 2003
posts:12555
votes: 3


The image I tested on was a 188 KB jpeg @ 3500 x 3500 without issue. Perhaps it is something in your code, a loop or otherwise that is eating up your memory? Try this simple test on the image in question to see if it displays in your browser (don't forget to edit the filename to your file):
<?php 
$filename = 'imagejpeg.jpg';
$im = imagecreatefromjpeg($filename) or die ("Cannot Initialize new GD image stream");
header ("Content-type: image/jpg");
imagejpeg($im);
imagedestroy($im);
?>
4:37 pm on Mar 22, 2007 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member whoisgregg is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Dec 9, 2003
posts:3416
votes: 0


This concludes to me that image dimensions matter more than image size.

Exactly correct. Although the source image you feed to imagecreatefrom(type) may be compressed, GD is creating an uncompressed version in memory for processing. GD will compress the image again when it is exported using image(type).

4:41 pm on Mar 22, 2007 (gmt 0)

Administrator

WebmasterWorld Administrator coopster is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:July 31, 2003
posts:12555
votes: 3


But on a 240 KB image? With 34 MB of memory available? That's one heckuva compression ratio ;) Seems something else is amiss.
4:41 pm on Mar 22, 2007 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member whoisgregg is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Dec 9, 2003
posts:3416
votes: 0


what are the alternatives to GD?

ImageMagick [imagemagick.org] is a popular alternative and can either be executed through system calls or by installing the ImageMagick extension [us3.php.net].

5:01 pm on Mar 22, 2007 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member whoisgregg is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Dec 9, 2003
posts:3416
votes: 0


During imagecreatefrom(type) calls, the memory usage is related to the number of pixels in the source image. So, it doesn't matter if you have 3500x3500 pixels which are all the same color or 3500x3500 pixels of random color noise, the memory allocation will be identical with GD.

You can test it by making two JPEGs which are of equal pixel dimensions but dramatically different file sizes then run each through this script.

<?php
ini_set('memory_limit', -1);
echo 'Start: '.memory_get_usage().' <br />';
$filename = $_SERVER['DOCUMENT_ROOT'].'/play/gd-memory/3500x3500-72.jpg';
$im = imagecreatefromjpeg($filename) or die ("Cannot Initialize new GD image stream");
echo 'ImageCreate: '.memory_get_usage().' <br />';
imagedestroy($im);
echo 'ImageDestroy: '.memory_get_usage().' <br />';
?>

I ran a 72 KB 3500x3500 JPEG and a 904 KB 3500x3500 image and both used up just under 62 MB of memory (61,392,088 bytes to be exact).

Added: I'm knee deep in a six month long image generation project so I've been tearing through books on ImageMagick, graphics programming with Perl, etc. to learn all the fundamentals. I used to spend a lot of time compressing my source images that would be processed by GD until I learned that the quantity of pixels exclusively determines memory usage. I was pretty surprised when I read that.

5:31 pm on Mar 22, 2007 (gmt 0)

Full Member

10+ Year Member

joined:June 24, 2004
posts:202
votes: 0


whoisgregg, out of curiousity, what have you chosen as your final solution to working with images in your project, GD, ImageMagik?
5:33 pm on Mar 22, 2007 (gmt 0)

Full Member

10+ Year Member

joined:June 24, 2004
posts:202
votes: 0


coopster, I tried the script you provided with the old memory limits and it failed. With new memory limits it processed ok.
5:45 pm on Mar 22, 2007 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member whoisgregg is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Dec 9, 2003
posts:3416
votes: 0


I use whichever is faster for any particular operation. I'm building the whole thing so any particular task is broken into it's component parts. So some parts will be done with GD, some with ImageMagick. So far, GD is faster most of the time. Some of the complex stuff is not supported by GD so it requires ImageMagick.

I'm still not sure why ImageMagick is slower, btw. It's supposed to be faster. But, even called from the command line directly (no PHP) the operations are slower. :/

I allocate 128M as default and the largest image buffer it generates is around 1,000 x 1,000 pixels (but I'll have multiple image buffers in memory at any given time). Final image sizes are around 500 x 500 pixels. I'll cache images to disk and use imagedestroy to clear those from memory so I don't bump into my memory limit.

6:11 pm on Mar 22, 2007 (gmt 0)

Administrator

WebmasterWorld Administrator coopster is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:July 31, 2003
posts:12555
votes: 3


I'm knee deep in a six month long image generation project so I've been tearing through books on ImageMagick, graphics programming with Perl, etc. to learn all the fundamentals.

I've never delved into graphics that intently (haven't had a need yet) but bought a book back in 2003 about PHP Graphics anyway. 99.99% of the graphics work here is performed using special software (read: not by me), but I have automated some things to make life easier. Therefore, you have my attention here and am eager to learn from you.

I have a jpeg of 188 KB @ 3500 x 3500 and ran it through your script there to have another look at memory usage to be certain I am not missing something.

Start: 17848 
ImageCreate: 61317304
ImageDestroy: 17992

Almost the same as yours. So, are we actually looking at the amount of memory being used by the GD library API's rather than image size? I wasn't certain so I ran it against a 3 KB image.
Start: 17896 
ImageCreate: 76552
ImageDestroy: 18040

Not according to that test. So, your statement in regards to ...

I used to spend a lot of time compressing my source images that would be processed by GD until I learned that the quantity of pixels exclusively determines memory usage. I was pretty surprised when I read that.

Where did you read that? I don't know enough about image processing but wouldn't mind picking your brain here if you don't mind sharing ;)

6:24 pm on Mar 22, 2007 (gmt 0)

Full Member

10+ Year Member

joined:June 24, 2004
posts:202
votes: 0


I'll second coopster's request!

Time is money however so I'll understand if you can't share your entire 6 month learning curve!

If I may add a concern from a PHP APPLICATION point of view. Cameras continue to take larger pictures while at the same time camera owners are becoming less familiar with the ability to scale the image resolution...

4:04 pm on Mar 23, 2007 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member whoisgregg is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Dec 9, 2003
posts:3416
votes: 0


Sorry, got busy yesterday afternoon. :/ I don't know which book specifically covered it (in fact, my expressing of it is certainly an amalgamation of various sources).

They all tend to have a first chapter that covers fundamentals of image processing. All image processing does is push around individual pixels and the only way to do that is to have all those pixels in memory. I'm pretty sure that technically the formula is

((pixel width * pixel height) * color depth)
(plus some storage overhead for the data).

The only reference online to this fact I could find is the GD FAQ [boutell.com] titled "Why does gd cause my PHP script to run out of memory?" (way at the bottom of the page)

Of course, opening truly huge images can cause real memory problems, if several are open at once. 8,000 pixels times 8,000 pixels times four bytes for truecolor equals a walloping 256 megabytes.

If you are wondering how programs can manipulate images which are larger than the computers memory (a task Photoshop deals with regularly), they only process one "chunk" of the image at a time, caching the results to disk as it goes. The image you see on screen is a composited scaled down version, hence why it takes a few seconds to redraw every time you zoom in (you may also be able to see the square "chunks" redrawing separately. GD has that capability as well. See imagegd2 [php.net], imagecreatefromgd2part [php.net].

Cameras continue to take larger pictures ... camera owners are becoming less familiar with the ability to scale the image resolution

Totally agree. Cheap consumer digital cameras are already bumping the 7-10 megapixel range and most users will cheerfully upload those huge files. Anyone serious about offering web services that include user image uploads should figure that any upload will initially need to be drastically resized and choose servers with enough memory to handle that operation.

I'll understand if you can't share your entire 6 month learning curve

I'll share as much as I can (and the parts I understand well enough to express). :)

5:14 pm on Mar 23, 2007 (gmt 0)

Administrator

WebmasterWorld Administrator coopster is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:July 31, 2003
posts:12555
votes: 3


Thanks for the insight, whoisgregg. I started on my own image manipulation class awhile back and I'll soon be revisiting it ... this time with some new knowledge to apply!

By the way, I just realized this last week and was going to post but forgot ... the GD Library has moved [webmasterworld.com]!

 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members