Clarification of spider process and inclusion time

just more clarification

[Clark]

I asked about this before, but still don't feel comfortable with the answer based on observation.

OK, when you are a new site, you get spidered by the deep crawl say early to mid April. Your site gets a PR by next update (my observation) or the following one (as it has been said here).

The following month, you may get a nice deep crawl. But in general the deep crawl shows up the update after next according to general wisdom.

From what I've seen, it looks more like it makes it in the next update. Is my observation wrong?

[jdMorgan]

Clark,

It looks like this to me:

Deep crawl 1
Wait 3-4 weeks
Update index w/deep crawl 1 data
Wait 0-7 days
Deep crawl 2
Wait 3-4 weeks
Update index w/deep crawl 2 data
Wait 0-7 days
Deep crawl 3
...

And then freshbot comes in and (apparently) randomly updates your listing every three days (or only once, or only a few times) and confuses everyone. Also, some here report a "late deep crawl" where sites are deep-crawled much later, but still appear in the next update.

Since PR is derived from incoming links, those links must be found before thay can count towards your pages' PR. If new incoming links miss the deadline for the deepcrawl, then they won't count until the next deepcrawl/update cycle.

HTH,
Jim

[markusf]

even if you get some links in time for the deepcrawl, they may not be displayed until after 2 updates. This is because all sites don't get crawled at the same time. I had some links from PR7 pages last update that where linking to me that got crawled by don't show up in my backlinks but show up when i search for my domain name grrr.