A website has a form that POSTS to
[kumo.swcp.com...]
When this is done from the browser the browser says it is a application/x-zip-compressed and ask to open/save.
The file returned depends on the form input so I am assuming it is a script of some sort.
I need to use a script to download the appropriate .zip file.
When I emulate the input of this form in my script and post it to the site the response object is a web page saying "sorry".
This message is also displayed if you use ACTION=GET for the form, or try to use a browser to fo directly to the website, etc.
When I copied the html form to my own web server and submitted via a browser it worked so the problem isn't me not being able to access the site from a different link.
I have specified that my request will accept the zip file type.
I think there must be some problem with my request, should I be requesting a file somehow not html? I couldn't figure out if this was the problem or how I would do this.
Please help me it's driving me crazy, I have included my script below.
The form that submits is at [kumo.swcp.com...]
my webpage with this form on it is at www.cse.unsw.edu.au/~phan447/download.html.
I have been given permission to use the data from the site above.
#!/usr/local/bin/perl
use LWP::Simple;
use HTTP::Request::Common "POST";
use LWP::UserAgent;
LWP::Debug::level('+');
# create user agent
my $ua = new LWP::UserAgent;
$ua->agent("Mozilla/8.0");
#set proxy
$ua->proxy('http','http://www-proxy.cse.unsw.edu.au:3128');
# URL
my $url = "http://kumo.swcp.com/stocks-bin/update.zip";
#Request
my $req = POST $url,[
'option' => 'Get Update'
'upd_length' => 'Weekly' ,
'format' => 'historical',
'username' => 'website',
'idate' => '0'
];
$req->header('Accept' => 'application/x-zip-compressed');
$req->content_type('application/x-www-form-urlencoded');
#print request
print $req->as_string();
#Response
my $res = $ua->request($req);
if ($res->is_success) {
print "sucess\n";
print $res->content;
}
else {
print "fail\n";
print $res->status_line,"\n";
}
exit;
-----output---------------------
LWP::UserAgent::new: ()
LWP::UserAgent::proxy: http [www-proxy.cse.unsw.edu.au:3128...]
POST [kumo.swcp.com...]
Accept: application/x-zip-compressed
Content-Length: 78
Content-Type: application/x-www-form-urlencoded
option=Get+Update&upd_length=Weekly&format=historical&username=website&idate=0
LWP::UserAgent::request: ()
LWP::UserAgent::send_request: POST [kumo.swcp.com...]
LWP::UserAgent::_need_proxy: Proxied to [www-proxy.cse.unsw.edu.au:3128...]
LWP::Protocol::http::request: ()
LWP::Protocol::collect: read 88 bytes
LWP::UserAgent::request: Simple response: OK
sucess
text/html<HTML>
<HEAD><TITLE>Sorry</TITLE></HEAD>
<BODY>
<H1>Sorry!</H1><BR><BR>
</BODY>
</HTML>
What you should do first, is to sniff HTTP header traffic in both cases and then compare the logs. I use Proxomitron's log window feature, but there many other tools that do it.
