Welcome to WebmasterWorld Guest from

Forum Moderators: ocean10000

Message Too Old, No Replies

ASP Link Crawler

Have looked for simple script all over the web...

4:07 pm on Apr 30, 2005 (gmt 0)

Junior Member

10+ Year Member

joined:Feb 27, 2003
votes: 0

I am looking for a simple asp-script, that can extract URLs from a website. Let's call it a sitemap crawler. I want to be able to feed a domain into the script, and the result should be a list of all pages on the website. It should achieve this by finding internal links on the default page, crawling them, enter the URLs into a database, crawl links on the next page and so on.

Anyone know if such a (free) script exists?

2:13 pm on May 1, 2005 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:June 13, 2002
votes: 0

Have you searched for asp spider? there should be plety to get you started.

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members