homepage Welcome to WebmasterWorld Guest from 54.161.147.106
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Code, Content, and Presentation / Perl Server Side CGI Scripting
Forum Library, Charter, Moderators: coopster & jatar k & phranque

Perl Server Side CGI Scripting Forum

    
Opening an external URL in PERL
Need some help with code snippet
trillianjedi




msg:3820937
 12:07 pm on Jan 7, 2009 (gmt 0)

Hi all,

Never coded in PERL before. I need to create a very small PERL script that simply opens a URL and then prints it. Actually I need to do a bit more than that but haven't got past this first base yet.

I found a few tutorials around on the web for PERL and ended up with this:-

#!/usr/bin/perl

print "testing\n";

$FilePath = "https://localhost/";

sysopen (FH, $FilePath, r);

print <FH>;

close(FH);

But it doesn't do what I intended it to do which is:-

1. Print to screen "testing" <- that bit works at least ;)
2. Open a URL residing on my local Apache-SSL service (in this case just the root index.html)
3. Print the contents of that page
4. Release the file handle.

I run it from the command line as:-

perl perltest.pl

It prints the "testing" OK but nothing else. No errors.

Can anyone help? Also, the perldoc website (where I found various functions) doesn't tell me what return type to expect from sysopen. What's the best way to examine the return for errors?

I'm guessing I may need some kind of "get" function, but a search for wget on the perldoc website doesn't reveal anything. Nor did a search for "get" reveal anything which looked right.

I don't really want to go through a full PERL tutorial as this is the first, and possibly last, time I am likely to need to use it and it'll be a 7 or 8 line script, tops.

Thanks!

 

trillianjedi




msg:3820944
 12:16 pm on Jan 7, 2009 (gmt 0)

Added - just realised I'm bound to need some kind of Secure Sockets library for PERL to get this working with HTTPS.

To keep it simple and enable me to continue, using regular HTTP would be fine for now...

trillianjedi




msg:3820953
 12:23 pm on Jan 7, 2009 (gmt 0)

OK, managed it. The perl.com website pointed me at the libwww-perl library which led me in turn to this code which now works:-


#!/usr/bin/perl

use LWP::Simple;

my $url = 'http://www.google.com/';

my $content = get $url;
die "Couldn't get $url" unless defined $content;

print $content;

Is the "unless defined" conditional here appropriate?

Thanks.

phranque




msg:3820955
 12:37 pm on Jan 7, 2009 (gmt 0)

you probably want to use some existing CPAN modules:
#!/usr/local/bin/perl

use LWP::UserAgent [search.cpan.org];
use HTTP::Request [kobesearch.cpan.org];

my $URL = 'http://www.example.com/';

my $agent = LWP::UserAgent->new(env_proxy => 1,keep_alive => 1, timeout => 30);
my $header = HTTP::Request->new(GET => $URL);
my $request = HTTP::Request->new('GET', $URL, $header);
my $response = $agent->request($request);

# Check the outcome of the response
if ($response->is_success){
print "URL:$URL\nHeaders:\n";
print $response->headers_as_string;
print "\nContent:\n";
print $response->as_string;
}elsif ($response->is_error){
print "Error:$URL\n";
print $response->error_as_HTML;
}

[edited by: phranque at 2:50 pm (utc) on Jan. 7, 2009]

trillianjedi




msg:3820968
 12:59 pm on Jan 7, 2009 (gmt 0)

Great Phranque - many thanks, that worked first time (well, second time - had a missing curly bracket but looks like you've edited now).

Thank you very much - that's got me underway.

phranque




msg:3820984
 1:12 pm on Jan 7, 2009 (gmt 0)

sorry i didn't notice your third post until now.
you can do the simpler version if you don't need the flexibility.
the "unless defined" conditional is good.

trillianjedi




msg:3820996
 1:18 pm on Jan 7, 2009 (gmt 0)

No, I prefer yours. Also I noticed that after I installed the libcrypt-ssleay-perl library HTTPS immediately started working off the bat (just by adding the S to the HTTP).

I'm starting to like PERL. Might have to experiment a little more with it.

Incidentially, how can I get at the $response->main_content_of_page ?

Other than error checking the headers, I don't really need them.

phranque




msg:3821062
 2:51 pm on Jan 7, 2009 (gmt 0)

i fixed my post to show the content - sorry about that - it's this line:
print $response->as_string;

trillianjedi




msg:3821182
 5:09 pm on Jan 7, 2009 (gmt 0)

print $response->as_string;

That seems to print headers and content for me. I did find it though (with a guess), its:-

print $response->content;

phranque




msg:3821579
 2:01 am on Jan 8, 2009 (gmt 0)

check out these modules - the objects you are playing with are these (sub)classes:
HTTP::Request
[kobesearch.cpan.org...]

HTTP::Message
[search.cpan.org...]

HTTP::Response
[search.cpan.org...]

witnesstheday




msg:3827234
 5:25 pm on Jan 15, 2009 (gmt 0)

Hi Phranque. I've been learning perl through hands on usage since about 2000 or so. I'm really "procedural" and not very object oriented, although apparently I have a flair for "functional" programming also. The most formal training in software I've had is a brief intro course to Linux I did. I'm going to sit down and start reading the blue camel book spock-style, shortly, but I wanted to know about this thing:


you probably want to use some existing CPAN modules:
#!/usr/local/bin/perl

use LWP::UserAgent;
use HTTP::Request;

my $URL = 'http://www.example.com/';

my $agent = LWP::UserAgent->new(env_proxy => 1,keep_alive => 1, timeout => 30);
my $header = HTTP::Request->new(GET => $URL);
my $request = HTTP::Request->new('GET', $URL, $header);
my $response = $agent->request($request);

# Check the outcome of the response
if ($response->is_success){
print "URL:$URL\nHeaders:\n";
print $response->headers_as_string;
print "\nContent:\n";
print $response->as_string;
}elsif ($response->is_error){
print "Error:$URL\n";
print $response->error_as_HTML;
}

What I want to know is how I go about using a cpan module. I suppose I would need to download something and install it on my server before I can USE LWP and USE HTTP?

If so, what is the best procedure for doing that? I run Apache/2.0.54 (Fedora) Server, which I bought maybe at most 3 years ago and it has whatever version of Perl was au fait at the time. It's on a nice juicy dell poweredge machine.

Maybe just reading the blue camel book from start to finish is going to make the biggest difference to my skill level, but I am keen to become properly proficient in using cpan, modules and packages properly, and indeed becoming the best perl programmer I can be, with as much ability in the field of OOP as is necessary. (Which is what?, by the way)

Hope you can help. I think I'm off to have some tea and cake and sit down with the blue camel book. I'll just go take it off my shelf right now. Terrible place for a book, a shelf. Needs to be open and in front of you - at least sometimes!

cheers

Shams

krugs




msg:3827243
 5:33 pm on Jan 15, 2009 (gmt 0)

Shams,

More reading for you mate:

[cpan.org...]

The Camel book also has a CPAN chapter, at least mine does (Third Edition)

witnesstheday




msg:3827267
 5:56 pm on Jan 15, 2009 (gmt 0)

Cheers. I started to notice this about the camel book just now as I sipped my tea! I'll stick with that for now, but I suppose I'll have to just dive into cpan.org sooner or later!

janharders




msg:3827271
 6:00 pm on Jan 15, 2009 (gmt 0)


What I want to know is how I go about using a cpan module. I suppose I would need to download something and install it on my server before I can USE LWP and USE HTTP?

yes. while many modules are probably already installed on your system, others may not. Those can be compiled (in case of c/xs-stuff) and installed
a) manually (and who'd want that)
b) using something like CPAN.pm (a perl module that does the installing for you ... run it with "perl -MCPAN -e shell" and just hack away -- might want to do that in a virtual machine or non-production environment in case anything breaks) and
c) using your favorite package manager. I'm a debian-guy myself, but I'm sure that fedora has many pre-packaged perl-modules as well. Use yum (that was the package manager on fedora, wasn't it?) and just do a search for, say, lwp. you should find a few perl-modules and you could just have yum download and install them.

If I do that on debian, I get a list like

jan@lsrv01:~$ apt-cache search lwp
libcrypt-ssleay-perl - Support for https protocol in LWP
libhttp-access2-ruby1.8 - HTTP accessing library for ruby
libhttp-cache-transparent-perl - Perl module used to transparently cache HTTP requests
libhttp-proxy-perl - A pure Perl HTTP proxy
liblwp-authen-wsse-perl - Library for enabling X-WSSE authentication in LWP
liblwp-dev - Light Weight Processes library (development)
liblwp-protocol-http-socketunix-perl - Perl module to speak http through unix sockets
liblwp2 - Light Weight Processes library
libwhisker-perl - Perl module geared for HTTP testing
libwww-freshmeat-perl - automates searches on Freshmeat.net
libwww-indexparser-perl - Fetch and parse the directory index from a web server
libwww-perl - WWW client/server library for Perl (aka LWP)

Say I wanted to install LWP::UserAgent, so I'd run

apt-get install libwww-perl

and have apt-get set it up for me. It's as easy as that, but there are many much better introductions. I should also add that there's an improved version of CPAN.pm, called CPAN Plus, but I don't use that yet.

witnesstheday




msg:3827273
 6:01 pm on Jan 15, 2009 (gmt 0)

ah, no wait - it just has references to CPAN but not a chapter with CPAN in the title - it's Second Edition.

The index has one page for CPAN, which is p377 in Chap 7, "The Standard Perl Library" - so looks like I'll have to read that link sooner not later. Cheers twice.

witnesstheday




msg:3827274
 6:03 pm on Jan 15, 2009 (gmt 0)

this sounds good - hopefully I can get my head round this all over coming weeks. cheers.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Code, Content, and Presentation / Perl Server Side CGI Scripting
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved