Welcome to WebmasterWorld Guest from 54.196.233.239

Forum Moderators: ergophobe

Message Too Old, No Replies

Indexing your content

web spiders, sitemaps, index

     

zpbs1914

3:31 pm on Aug 2, 2006 (gmt 0)

10+ Year Member



Does anyone know of:

1. A reliable spider that will INDEX ONLY (don't need full web download) a site over HTTP:

2. A tool with a method to capture the results of site index search (such as site:www.mydomain.com in Google)into a text file.

I've tried close to about 10 tools over the past 3 days and all of them are less than reliable. This is my last effort before I look to grab the code and write my own. Any info would be appreciated, thanks

 

Featured Threads

Hot Threads This Week

Hot Threads This Month