tedster - 8:31 am on Sep 20, 2012 (gmt 0)

Here's my theory. The Google back end works with seed websites of various kinds. There are "tables" of very high trust sites, very low trust sites, sites that line up with different qualities that Google works with.

These "tables" contain only the hostnames, and not the rest of the URLs. And those hostnames need to be imported into the full index right when it goes live - and that means imported to a lot of server clusters, not just one.

Google moves a LOT of data from server cluster to server cluster, during development and when going live. In fact, one "server" can better be understood as a building full of server racks. At Google's scale, and with the Caffeine infrastructure, one physical machine is just not a realistic unit.

Sometimes those data ports on the back end get buggy - it happens. Partial data imports, corrupted entries, etc. And then sometimes a good site can lose its home page (its hostname) for a while, because of a data problem that occurred with the "good site data import".

Or so goes my theory.

A lot of reading and data study has gone into it, but I sure can't claim it's verified. The most I can say is that during the time where we focused on watching the various data centers, some members here are sure they saw it happen in real time. And some members saw their home page disappear from the google.com SERPs but not from Google's other ccTLDs.