Welcome to WebmasterWorld Guest from 126.96.36.199
Forum Moderators: phranque
In my dreams, this would be done by a nice piece of software into which I feed all the top level domains. It then goes off and spiders the hell out my list, producing one big list of the URLs covering every site and every page within.
Is this just a dream, or is there something out there to so this.
BTW I don't have all these site on my server, they are scattered to the wind!
Perhaps Xenu [home.snafu.de] might be a better choice. This is a link checker that ultimately gives you a list of URL in the site and a site map. You would have to do each site seperately but you could then save each report as an html page and build your own mini directory.
Also you can save each site's file for updating at a later stage.
And Free as well...