Forum Moderators: coopster & phranque

Message Too Old, No Replies

Perl/PHP help.

How do I grab a remote web page and stick it in a file?

         

uplate

3:40 pm on Mar 28, 2003 (gmt 0)

10+ Year Member



Hi, I want to automatically update a file of spider IP addresses on my server in Perl or PHP but I'm not sure if it's possible or where to start. The data resides on another site. (I check it daily & do it by hand if necessary but I need a paranoia-free holiday). Should I be looking at sockets or is there a easier way? Any pointers would be appreciated.

SinclairUser

4:04 pm on Mar 28, 2003 (gmt 0)

10+ Year Member



Uplate,

You can use this Perl module LWP::UserAgent to get the data from the other site and then parse the result to extract your data.

use LWP::UserAgent;
$ua = new LWP::UserAgent;
$ua->agent("$0/0.1 " . $ua->agent);
# $ua->agent("Mozilla/8.0")
$req = new HTTP::Request 'GET'=> 'http://www.sn.no/libwww-perl';
$req->header('Accept' => 'text/html');
# send request
$res = $ua->request($req);
# check the outcome
if ($res->is_success) {
print $res->content;
} else {
print "Error: " . $res->status_line . "\n";
}

Chris.

RonPK

5:47 pm on Mar 28, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



If you prefer PHP, try something like this:

$fp = fopen("http://someurlhere", "r");
while (!feof ($fp)) {
$contents .= fgets($fp, 4096);
}
fclose ($fp);

$contents now contains the requested file and can be parsed as you wish.

uplate

8:54 pm on Mar 28, 2003 (gmt 0)

10+ Year Member



Sorted! Thanks for that.