Trouble downloading file using LWP

Need Help

[aerpam]

I am trying to write some code to download files overnight for later use. I've tried the LWP mirror function, and get functions. They both are unreliable - sometimes they will download the whole file, sometimes not. Here is what I am currently trying:

$url = "http://ichart.finance.yahoo.com/table.csv?s=" . $stock_symbol . "&d=11&e=27&f=2005&g=d&a=2&b=26&c=2002&ignore=.csv";

$ua=LWP::UserAgent->new;
$ua->agent("MyApp/0.1 ");
my $req = HTTP::Request->new(GET => $url);

# Pass request to the user agent and get a response back
my $res = $ua->request($req);

# Check the outcome of the response
if ($res->is_success) {
print CSVFILE $res->content;
}

Thanks,
Aerpam

[bennymack]

My suggestion would be to loop until the request is successful.

I would also print out any errors that are encounterd and look into a way to increase the timeout if that is an issue.

[aerpam]

bennymack,

Thanks for the reply, I do however have a check to see if it is successful, and print to my local file only if it is. Unfortunately this frequently only yields a partial download of the file. Do you know if there is any way to tell if I have the complete file?

Thanks,
aerpam