Spiders

how can I make one / find one (cgi or perl)

[luckynh]

I have been looking for a cgi / perl spider that will spider any inputted url and follow links within the site.

I just set up MySql on my Linux Box and I would like to spider my clients sites and enter the all the gathered info (meta data,title,alt,100%text)into MySql Database.

I would like to evaluate a demo if someone can point me to the right place.

[littleman]

There is a turnkey example at [xav.com...] . It is a very good script and it is a very good value.

[luckynh]

Thanks littleman.

I'll check it out and let ya know

Lucky

[Brett_Tabke]

If you have installing perl modules figured out, download the Robots perl module. Inside is a script called "poacher" that works great.

[luckynh]

Thanks Brett
Do you have a link?