Forum Moderators: phranque

Message Too Old, No Replies

Sitemap Software/Code

A spider that just lists the links

         

aspdaddy

12:31 pm on Apr 14, 2003 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



Maybe someone here can help with this. I'm trying to find/write a program that produces a list of all a websites internal links in this format:

a b
a c
b c
etc.

a,b,c are absolute urls.
a links to b, a links to c etc.

I need it to run on IIS or run locally under windows. Have tried Zenu & W3CLinkbot, but cant get the output in that format.

I have written a simple spider but ASP has no url class & it is difficult to work out the urls of the paths and default files for sites with many levels. Also hard to debug without crashing the server :(

Anyone know a program or some code that can do it?
If anyone can write it pls sticky me.