Deep Crawling?

Deep crawling large sites?

[8cool]

Does anyone have an idea how long it takes googlebot to deep crawl a large site with 24K static html pages?. Will it be couple of weeks?

[seindal]

It depends on many things, including how many and how good incoming links your site has, and how long it takes googlebot to discover those off-site links. The more and the better incoming links you have, the more often googlebot will pass, and the larger chunks it will take at a time.

I'd say you have to wait several months before the whole site is indexed.

René

[troels nybo nielsen]

Welcome to WebmasterWorld, 8cool.

Other factors of some importance are your internal linking structure, the size and complexity of your html code and the length and complexity of your filenames. It's always good to make life more easy for googlebot. Two or three unnecessary complications for googlebot will not really make any difference on ONE page, but with 24k pages it sums up.

[dirkz]

Welcome to WebmasterWorld, 8cool.
With so many pages I wouldn't even expect Googlebot to crawl all of them or even put them in the index. Unless of course you have a high PR.

I would guess with a PR9 it would take less than 4 weeks to have all pages indexed :)