Google Un-Updates?

Trying to figure out how updates may (or may not) work now....

[jude2]

I've only been at this a couple of months, so by the time I got the hang of the dance, the dance has, apparently, died [webmasterworld.com]. I don't show up for my main keywords yet, so I don't know whether this is a good thing or a bad thing for me...(any guesses?) But my main question is this:

I changed my index page during the first week of August. Until last week, the old page was the one in Google's cache. Then, last week, the new one was the cached page, and I was thrilled. Now, the old one is BACK as the cached page - and I seem to have dropped in the index. I don't play any games or try any tricks...any idea what happened? Why would they re-cache the older one when they HAD the new one? What don't I understand?

Any help would be MUCH appreciated!

:)
Jude

[takagi]

Going back to an older cache is not so uncommon for Google. You could see it like this: Google has a full index (the one that used to be updated once a month) and a smaller database with recently spidered pages. This smaller one is like a circular buffer (First In, First Out). If a cache is requested for a page that can be found in this smaller database, then Google will send the more recent cache. This smaller database has only a limited page capacity otherwise it would take a long time to search it (which would make Google slower for the visitors). With newer pages getting into this database, older ones will be dropped. Once dropped, Google will use the words found on the older version of the page and send an older version of the cache (the one from the full index). In some cases however, the cache and page content used by Google could be out of sync (e.g. the snippet shows new text, but the cache is still old).

[MonkeeSage]

takagi:

So there was a deep crawl recently?

Jordan

[takagi]

So there was a deep crawl recently?

I'm not sure if you can still talk about a 'deep crawl'. But the deep crawl was not related to the 'fresh listings' with or without the fresh tag (the date next to the URL) in the SERPs. The results of the 'deep crawl' were used for the monthly update whereas the data from the fresh bot was often discarded. But that was true until the start March. GoogleGuy confirmed that the deep crawl data collected in April was not used for the Dominic update.

[moreIdeas]

Deep Crawl Happening as I type

[johnnydequino]

What deep crawl? Googlebot just comes to my home page every day and caches the changes. No new pages added in weeks. Maybe it's back to the monthly google dance? (Grrrrrr - frusterating)

jd

[MonkeeSage]

takagi:

Deepbot is the one that round-robins, as I understand things, not freshy, that's why I asked.

Jordan

[takagi]

No it's quite the opposite. But I didn't want to introduce the term 'fresh bot' and 'deep bot' in message 2 since it is not really clear how the behaviour of the bots is.

[MonkeeSage]

takagi:

I was under the impression that the 'deep crawling bot' (for lack of a better expression) had physical constraints on how much data it could hold, and used a circular buffer for this smaller database it was building from the crawl; and that the 'fresh crawling bot' (for lack of a better expression), reported home often enough not to need a circular buffer. Is that incorrect?

Jordan

[2_much]

It sounds like freshbot behaviour to me. Jude2, this is part of Google's new pattern, with their emphasis on freshness. Googleguy gave freshbot a new name - deepfreshbot. My understanding is that although there are still monthly deep crawls, Google's relying more and more on their freshbot to provide the content used for rankings. So you might be seeing the effects of this.

[jude2]

Thanks, 2much. My site has been crawled (deeply, by whichever one it is!) since the old page was cached, so I couldn't figure it out. The new version showed up today, though, along with all of my pages (for the first time)...I guess there's never really anything to do but wait? Patience, I know..

Thank you!