Simulate Form Post Using Perl

They say it cant be done...

[PhraSEOlogy]

I have tried to find a way to create a perl script that can simulate the (exact same) action of someone filling out a form on a webpage and submitting the form and then the browser being redirected to the target page.

Now I can use LWP::UserAgent to simulate the post of the data but then the results are returned to the script. What I want to happen is to get the script to submit the form and then allow the browser to be redirected to the target URL - in the same way they would if they did it from a webpage.

Is this possible with PERL?

[andrew_m]

Is not $ua->post(...) the thing you need? The syntax is the same as $ua->get(...).

Have a look also at HTTP::Request -- that's what's working behind the scenes.

[PhraSEOlogy]

Heres the problem.

Form on webpage
----------------
User enters data and clicks submit.
Data is sent to action URL.
Action URL displays result in browser window.

Form simulated in perl
----------------------
Script executed and enters data and submits.
Data is sent to action URL.
Action URL returned to perl script.

How do I get the Action URL to return the results to the browser and not the perl script?

[andrew_m]

How is it different from any other data you work with? Once you got the results treat them as data and display to the browser the same way as if you've generated that data yourself -- i.e. wrap them into a query, show in HTML or whatever else.

But then again, may be I still don't get your question.

[PhraSEOlogy]

andrew,

I am having difficulty trying to formulate the question exactly.

Let me give you another example and it might clear things up.

Lets say we have a search box form on site-A for a major online book seller site-B. If the user fills out the form in his browser and clicks submit then he will be transferred to site-B when the form has been processed. The results are shown on site-B.

Now if the search box executes a script on site-A that submits the form to site-B (using something like LWP) the results are returned to the script. Printing the returned HTML will display correctly but the browser is still pointing to site-A. Any relative links in the HTML will fail as the browser is not pointing to site-B. I.e the browser never got pointed to site-B as it would have been using a real html form.

[andrew_m]

Well, my previous answer still stands -- you got results from site B. In this case the result is the HTML code.

Now, if you simply redisplay that to the user you first of all likely to violate site B's copyright and you have a problem of related links that you already aware of.

What to do next depends on your intent.

a) You can analyze the HTML, strip it down to just data -- prices, descriptions and what not, and then display that to your visitor in your layout. This can be white hat or black hat -- I do that a lot, but I have contracts with all my "site Bs".

b) you can redirect your visitor to the site B. You cannot redirect using POST, so you'll have to re-map that into a GET -- most sites will accept both on the same form.

c) you can have a piece of javascript on the site A that will submit form to site B as normal, but right vefore submitting it will call a cgi-bug on your site so that you know what your visitor submitted.

..man that's long, I have too much time today :)

[PhraSEOlogy]

Thanks for the info.

I have to use post - site-B forms all use POST - GET will be not be processed.

I dont want to use the returned html - I really do want the visitor redirected as they would using a real html form.

So I guess the answer is - it cant be done with the restrictions I have.

[Moby_Dim]

Haven't you tried to use Location...? (or I have not understood the problem.)

[rocknbil]

GET will be not be processed

If this is true, look into how you are reading and parsing incoming data. A get should parse as easily as a post, it's just it is limited to the length of the query string.

But I think your answer lies in the other, now corrected, thread.

[PhraSEOlogy]

Rocknbill,

Regarding GET and POST - I meant that site-B will only process POSTed data from the form. You can use the GET method but it just generates an error from site-B.

I can POST or GET data using curl or LWP no problem. The real problem here for me is getting the browser redirected to the forms target site after the perl script submits the form.

[volatilegx]

PhraSEOlogy,

After your CGI script POSTs the form data, a reply is sent to the CGI script. Your CGI script merely has to discard the reply and use a Location: header to send the browser to the website you want. Here's a quick and dirty example:

use HTTP::Request::Common qw(POST);
use LWP::UserAgent;
$ua = LWP::UserAgent->new;
my $req = (POST 'http://www.example.com/cgi-bin/process.cgi',
 ["PARAM1" => "foo",
 "PARAM2" => "bar",
 "PARAM3" => "coo",
 "PARAM4" => "laa"]);
$response_to_discard = $ua->request($req);
print "Location: http://www.example.com/whatever.html\n\n";
exit;

[PhraSEOlogy]

This is my solution to the problem.

Thanks to volatilegx for help with the code and preventing me from overcomplicating the solution.

#!/usr/bin/perl

use LWP::UserAgent;
$ua = LWP::UserAgent->new;
$ua->agent("Mozilla 8.0 blah...");

use HTTP::Request::Common qw(POST);

my $req = (POST 'h**p://www.url-to-post-to',
["param" => "value",
"param" => "value"]);

$request = $ua->request($req);
$content = $request->content;

print "Content-type: text/html\n\n";
print "<base href=\"h**p://www.target_site_url\">";
print $content;
exit;

Although the users browser still points to my site the <base> tag resolves the broken links that would otherwise prevent this from working.