Welcome to WebmasterWorld Guest from

Forum Moderators: coopster & jatar k

Message Too Old, No Replies

Extracting new content from a website

11:37 am on Nov 5, 2003 (gmt 0)

Full Member

10+ Year Member

joined:Aug 29, 2003
votes: 0


I'm looking for a script that :
- spiders a few pages for new links. The script has to follow these links 2 levels deep.
- saves some of the content of these pages (I need just a few chunks of text which can be easily parsed) into some format
- compares this file with an existing mysql database.

I think php is the best language to write this script, but I was wondering if there are scripts which are already doing this? If not, can anyone help me develop this?


12:18 pm on Nov 5, 2003 (gmt 0)

Junior Member

10+ Year Member

joined:Feb 26, 2003
votes: 0


I've not seen anything available that will do this, however the Snoopy php class [snoopy.sourceforge.net...] would be a good place to start this project.

It simulates a web browser and has a method for fetching links.

Hope this helps


Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members