Forum Moderators: open
1. A reliable spider that will INDEX ONLY (don't need full web download) a site over HTTP:
2. A tool with a method to capture the results of site index search (such as site:www.mydomain.com in Google)into a text file.
I've tried close to about 10 tools over the past 3 days and all of them are less than reliable. This is my last effort before I look to grab the code and write my own. Any info would be appreciated, thanks
