Welcome to WebmasterWorld Guest from

Forum Moderators: coopster & jatar k

Message Too Old, No Replies

PHP Crawler



9:16 am on Sep 7, 2002 (gmt 0)

10+ Year Member

I am looking for a PHP crawler, I know Perl has a similar package but never heard of a PHP one.

Notes on how to make one are also welcome.

I already read this thread:

brotherhood of LAN

9:59 am on Sep 7, 2002 (gmt 0)

WebmasterWorld Administrator brotherhood_of_lan is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

Hello Martin,

I guess its a stupid question I'm asking, but, does it have to be PHP? I'm making a wee assumption you want it dumped into mySQL.

If that's the case - then FDSE can do that.

If it's not....I'm on the same hunt as you....well...I will need something along the same lines.

If nothing is found I'll no doubt have to pay someone to make one.......will be stickying jatar_k if he has the time ;)


10:20 am on Sep 7, 2002 (gmt 0)

10+ Year Member

>If that's the case - then FDSE can do that.

Thank God Google know what is FDSE.

I saw a JavaScript PageRank calculator, but it requires lots of work to get the page links, etc.

I decided to make something similar in PHP, so it can save me time. I don't want the pages indexed - just the site structure.


Featured Threads

Hot Threads This Week

Hot Threads This Month