Web Automation Using LWP::UserAgent

[hemant7]

Champs .. I just want to know that how to pass values using LWP:UserAgent .. I mean how i can automate that page .. I can automate eveything with Mechanize but Mechanize does not supports javascript .. SO want to learn using with LWP::UserAgent .. I am writing my code as below ..

!usr/bin/perl -w
use strict;
use LWP::UserAgent;
use HTTP::Request;
use HTTP::Response;

my $lwp = LWP::UserAgent->new();
my $req = POST 'http://www.example.com', [ city => CityName', q => 'Widget' ];
my $content = $lwp->request($req)->as_string;
# Printing Content ..

[edited by: phranque at 7:41 pm (utc) on April 22, 2009]
[edit reason] exemplified url & search terms [/edit]

[krugs]

See your other thread.

[phranque]

welcome to WebmasterWorld [webmasterworld.com], hemant7!

seems like you are close.

you should look into the LWP::UserAgent REQUEST METHODS [search.cpan.org] documentation for the post method.

you can learn more details and alternate methods here:
HTTP::Request::Common - Construct common HTTP::Request objects - search.cpan.org [search.cpan.org]

also, maybe it's a typo in the post but make sure your first line looks more like this:
#!/usr/bin/perl -w

[hemant7]

But its not going upto its way ..
I mean i am unable to post a page .. Whenever i tries to post a page, my page remain to the same page .. vars is not posting/passing to next page ..

[phranque]

the POST is a request.

if you want some form parameters valuated in the response page, it is up to your script to do that work.

[hemant7]

Can anyone give me the snippet for vars passing from a form with LWP::UserAgent ..
Ok .. What i want to do is ..

I want to crawl a site .. But i am unable to do it using LWP::UserAgent ..
I am writing a small snippet here .. My site is having one city box, one search box, one area box and one button ..
So, I had written code to automate it like this ..

#!/usr/bin/perl -w
use strict;
use LWP::UserAgent;
use HTTP::Request;
use HTTP::Response;

my $lwp = LWP::UserAgent->new();
my $req = POST 'http://www.example.com', [ city => CityName', search => 'Widget', area=> 'area' ];
my $content = $lwp->request($req)->as_string;
# Printing Content ..

Now content I need is whatever came after posting the form to next page .. I mean after filling all the boxes and submit the form .. Getting me ..

Back to me ..
Cheers ..

[chorny]

my $res = $lwp->request($req);
if ($res->is_success) {
my $content=$res->content();
} else {
die $res->status_line;
}