incrediBILL - 6:48 pm on Jul 14, 2006 (gmt 0)

Is there a reason it cannot use the previously-collected Slurp dataset? Or the same crawler?

I'm 100% behind this sentiment as there are ZERO reasons that we have to subject ourselves to more than one Yahoo! crawler. They should all share a common data set internally and not continue to burn up our bandwidth over and over and over.

If that common data is too old for the task that wants it, fine, queue it up for a refresh crawl, but only ONE refresh crawl, not FIVE.

Just my $0.02 worth.