I need something like the Sim Spider

I need a spider simulator

[Randyd]

Does anyone know if I can get a spider simulator like the Sim Spider on WebmasterWorld. I would like to offer my visitors something like that. So they could put their URL in the http field and have it output a list of pages found on their site or the site they want to spider.

Thanks

[Matt Probert]

Comes as standard with Perl. Check the "Perl Cookbook" by O'Reilly for details of how to quickly and easily write a Perl script to spider a URL.

Matt