Hanu - 3:21 pm on Sep 24, 2004 (gmt 0)

>On the other hand the crawler depends on additional URLs that indexer gets by analysing crawled data.

If that's true, there is not much left to do for the crawler, is there? No, the crawler filters new URL's out on its own. Also it is fed existing URLs from the database and not the indexer.

>and with all that computing power they have the bottleneck is bound to be in "bandwidth" area

Word-based indexers are computationally expensive. That's what the G needs the computing power for. You think they buy all this machines and then force them to idle by putting stops into the code? No way! The indexer is the bottleneck, not the crawlers.