ashear - 11:13 pm on Dec 13, 2006 (gmt 0)

Age of the site makes a huge factor.

I have brand new sites with millions of URL's that hardly get crawled.

I have old sites with millions of pages that get crawled millions of times a day.

I have small new sites that have great content and take forever to get indexed and I also have small old sites with useless content that get crawled to death.

It seems to be about the age of the site, possibly the popularity could play a large part of the factoring as well.