homepage Welcome to WebmasterWorld Guest from 54.204.94.228
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Code, Content, and Presentation / Perl Server Side CGI Scripting
Forum Library, Charter, Moderators: coopster & jatar k & phranque

Perl Server Side CGI Scripting Forum

    
Script stalls when uploading 500k+ image
But 200k image uploads in 2 seconds
MichaelBluejay




msg:4186291
 3:47 am on Aug 13, 2010 (gmt 0)

I can upload a 200k image in a mere 2 seconds. But if I try to upload a 500k image, my browser tries to submit the form for several minutes. It would probably time out eventually but I haven't waited around that long. What gives? Here's my pared-down test code:
#! /usr/bin/perl

use CGI;
$cgi = new CGI;
@fields = $cgi->param; #gets the names of all form fields
$upload = $cgi->param('file');
open (UPLOAD, ">$upload");
while ($chunk = read($upload, $data, 50000) ) {print UPLOAD $data;}
close (UPLOAD);
print "Location:index.html\n\n";

 

phranque




msg:4186362
 7:03 am on Aug 13, 2010 (gmt 0)

you might need to increase the default maximum POST size specified by the CGI module.
the variable used for this is $CGI::POST_MAX.
you can get more information in the Avoiding Denial of Service Attacks [perldoc.perl.org] section of the CGI module documentation.

i think you will find as a practical matter that you will often have various problems when you get to the 100M level.
due to typical upload speeds while using http protocol many connections will time out well before a 500M file is uploaded.

added: you may also run into problems if memory is used to temporarily hold the file data during the post/upload.
note that some of these details may have changed - it's been a few years since i dug deeply into this code.
obviously some of these issues will be affected by the network connection as well as the platform-specific implementation of the CGI module on the server.

MichaelBluejay




msg:4186706
 6:50 pm on Aug 13, 2010 (gmt 0)

Thanks, setting $CGI::POST_MAX did the trick!

$CGI::POST_MAX = 1024 * 1000; # allows uploads up to 1Mb


Thanks again!

rocknbil




msg:4186708
 6:54 pm on Aug 13, 2010 (gmt 0)

Well, I have some "old code" I've never had a reason to change. This code opens a filehandle for **both** in and out. The only thing hinky I see is I question why your buffer is so large, 1024 should suffice. It doesn't speed anything up making it larger. The other thing I've grown accustomed to, and found that, is that $buffer and $bytesread are globals used by the cgi module and you're using different variable names (this may have been changed, it always bugged me so it make sense that it has.)

For your perusal and experimentation:


# $outfile = full path to file to write
#infile is your $upload
# error is my custom error handler sub
# binmode generally only needed on Windoze, and sometimes not then
open(OUTFILE, ">$outfile") || &error("Cannot open $outfile $!");
# binmode OUTFILE;
while ($bytesread = read($infile,$buffer,1024)) {
$totalbytes += $bytesread;
print OUTFILE $buffer;
}
close($infile);
close(OUTFILE);


I've never had to mess with the file size vars in the CGI mod.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Code, Content, and Presentation / Perl Server Side CGI Scripting
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved