nube - 9:20 pm on May 21, 2001 (gmt 0)

everyman, when you say deep, would this also include subdomains as part of the main domain or does the spider view subdomains as separate new domains which it has to crawl,.
Basicaly is a subdomain treated as a new domain,
Alsoi think you mentioned your site is in the 100k range for documents, how far does the spider crawl in that many documents before it cuts off.
thanks nube