homepage Welcome to WebmasterWorld Guest from 54.227.25.58
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Subscribe to WebmasterWorld

Home / Forums Index / Code, Content, and Presentation / Perl Server Side CGI Scripting
Forum Library, Charter, Moderators: coopster & jatar k & phranque

Perl Server Side CGI Scripting Forum

    
Web Automation Using LWP::UserAgent
hemant7




msg:3897578
 8:37 am on Apr 22, 2009 (gmt 0)

Champs .. I just want to know that how to pass values using LWP:UserAgent .. I mean how i can automate that page .. I can automate eveything with Mechanize but Mechanize does not supports javascript .. SO want to learn using with LWP::UserAgent .. I am writing my code as below ..

!usr/bin/perl -w
use strict;
use LWP::UserAgent;
use HTTP::Request;
use HTTP::Response;

my $lwp = LWP::UserAgent->new();
my $req = POST 'http://www.example.com', [ city => CityName', q => 'Widget' ];
my $content = $lwp->request($req)->as_string;
# Printing Content ..

[edited by: phranque at 7:41 pm (utc) on April 22, 2009]
[edit reason] exemplified url & search terms [/edit]

 

krugs




msg:3897837
 3:53 pm on Apr 22, 2009 (gmt 0)

See your other thread.

phranque




msg:3898357
 7:27 am on Apr 23, 2009 (gmt 0)

welcome to WebmasterWorld [webmasterworld.com], hemant7!

seems like you are close.

you should look into the LWP::UserAgent REQUEST METHODS [search.cpan.org] documentation for the post method.

you can learn more details and alternate methods here:
HTTP::Request::Common - Construct common HTTP::Request objects - search.cpan.org [search.cpan.org]

also, maybe it's a typo in the post but make sure your first line looks more like this:
#!/usr/bin/perl -w

hemant7




msg:3899295
 5:50 am on Apr 24, 2009 (gmt 0)

But its not going upto its way ..
I mean i am unable to post a page .. Whenever i tries to post a page, my page remain to the same page .. vars is not posting/passing to next page ..

phranque




msg:3899343
 7:37 am on Apr 24, 2009 (gmt 0)

the POST is a request.

if you want some form parameters valuated in the response page, it is up to your script to do that work.

hemant7




msg:3899357
 8:01 am on Apr 24, 2009 (gmt 0)

Can anyone give me the snippet for vars passing from a form with LWP::UserAgent ..
Ok .. What i want to do is ..

I want to crawl a site .. But i am unable to do it using LWP::UserAgent ..
I am writing a small snippet here .. My site is having one city box, one search box, one area box and one button ..
So, I had written code to automate it like this ..

#!/usr/bin/perl -w
use strict;
use LWP::UserAgent;
use HTTP::Request;
use HTTP::Response;

my $lwp = LWP::UserAgent->new();
my $req = POST 'http://www.example.com', [ city => CityName', search => 'Widget', area=> 'area' ];
my $content = $lwp->request($req)->as_string;
# Printing Content ..

Now content I need is whatever came after posting the form to next page .. I mean after filling all the boxes and submit the form .. Getting me ..

Back to me ..
Cheers ..

chorny




msg:3912945
 8:52 pm on May 13, 2009 (gmt 0)
my $res = $lwp->request($req);
if ($res->is_success) {
my $content=$res->content();
} else {
die $res->status_line;
}
Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Code, Content, and Presentation / Perl Server Side CGI Scripting
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved