Oh, this wretched machine that raises me to such highs and then pulls the magic carpet out from under my feet of clay... or is it that I fly to close to the sun and my melted wings cause me to plummet to the earth ... well whichever metaphor you choose.
Freshbot has been indexing our site all this last month, four time our freshened and cached homepage appears at the tops of SERPs. Hits and orders triple. All is right with the universe. And then a few days later, <PooF>, like Keyser Soze, vanished from existence. The only things left over are old cached pages like a body in the desert pick over by vultures.
Needless to say that with no updated optimized index page, sales fall off, bridges are lept from, and widows are left to collect the insurance money. All of this, only to happen all again a few days later like some causal time loop instigated by a massive energy fluctuation in a rift in the space/time continuum.
When will the madness end? Will this stop after the update? Will post update rankings be similar to the freshbot ones?
When you get a page from a site, it also returns the last modified date for that page. The next time googlebot requests the page is sends "get if-modified-since" the date that was returned the last time it got the page. If it has not been modified, your server will return the status code "not modified" and google will use the old version. It is a way of saving bandwidth.