homepage Welcome to WebmasterWorld Guest from 54.147.214.74
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Code, Content, and Presentation / Perl Server Side CGI Scripting
Forum Library, Charter, Moderators: coopster & jatar k & phranque

Perl Server Side CGI Scripting Forum

    
Grab source code from any supplied URL
There must be an easy way to do this...
skube

10+ Year Member



 
Msg#: 3354 posted 9:51 pm on Dec 3, 2003 (gmt 0)

Hi, I'm would like to have a form with a field where the user supplies any URL. Clicking submit then grabs the source of the page specified in the URL.

Does anyone know the basic syntax for this? I'm gathering one uses the CGI module...

Thanks,
skube

 

DrDoc

WebmasterWorld Senior Member drdoc us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 3354 posted 10:29 pm on Dec 3, 2003 (gmt 0)

You don't have to use the CGI module. Have you looked into LWP?

skube

10+ Year Member



 
Msg#: 3354 posted 11:28 pm on Dec 3, 2003 (gmt 0)

Ah right, thanks Dr. Doc...so something like this?

use CGI;
use LWP::Simple;
$query = new CGI;

$webpage = $query->param("web_url");
my $source = get $webpage ;

skube

10+ Year Member



 
Msg#: 3354 posted 11:36 pm on Dec 3, 2003 (gmt 0)

But now I have another problem, I have the source of the URL stored in the $source. However, I wish to convert each source line into an array @source, to do further manipulations.

Usually, one would open a file and assign each line using <> as below:

open(DAT, $data_file) ¦¦ die("Could not open file!");
@raw_data=<DAT>;
close(DAT);

But for my case, I don't have a file which I'm opening, just a very long variable $source. So, I want to do something like:

@source = <$source>

But obviously, that is incorrect syntax.

sugarkane

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 3354 posted 12:25 am on Dec 4, 2003 (gmt 0)

Probably best to use the split function :-

@source=split(/\n/, $source);

That will give an array with one line of code per entry.

(NB that should work for pages stored in *nix format. For Windows files you might have to use a more complicated regex to deal with the \r carriage returns as well as the \n newlines...)

divaone

10+ Year Member



 
Msg#: 3354 posted 4:09 pm on Dec 4, 2003 (gmt 0)

Why not use a script? Try http,//willmaster.com/master/snooper/

You will need the LWP::Simple mod, and probably already have it installed.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Code, Content, and Presentation / Perl Server Side CGI Scripting
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved