How many Google indexes?

[lucy24]

Found a slight head-scratcher and wondered if anyone can shed light.

Yesterday's logs show a visit to an obscure but recently updated page. In fact I had the googlebot re-crawl it just a couple of days earlier. Search string-- mercifully visible-- made it unambiguous that they were looking for the new material.

Out of curiosity I plugged the same search string into google ... and came up cold. If I try individual words (the search isn't in English) I get a couple of hits on other sites. Not mine.

The request was from an iPad so I tried my own, both with the app and through google dot com in the browser. (This involves cutting-and-pasting the text, since it can't be typed directly-- but this is undoubtedly what the original searcher did too so that's not an issue.) Still nothing.

I Am Mystified. IP points to my general geographical region, and definitely the same state. Are there multiple googles covering the same area?

[JD_Toims]

Are there multiple googles covering the same area?

Yeah, it used to be you could visit data centers independently [I'm guessing you missed the days of "watching the dance" on sites like McDar] via IP Address, but then Google switched their internal load balancing and you can hit any of a number of data centers [which may or may not be aligned and serving the same results] for any given query to any given data center. You can even be switched to a different one on a "next page" click, so there's really no telling what's coming from where or exactly how many different indexes they're "playing" with any more.

[lucy24]

Whew. So it may even take longer for an index to populate the different data centers than it takes to create the index in the first place.

Things are back to normal today. Pity there's no way of knowing whether it's a change in data centers or an update to the original one. But in the course of checking I noticed another nice detail.

Search for:

wordone wordtwo wordthree

without quotation marks. Result shows (matching the order of items on the page):

first-occurrence-of-word-one in context ... first-occurrence-of-word-two in context ... "wordone wordtwo wordthree" phrase

Hey, nice going, search engine. If I were a human performing this search, that could really be useful. It could also eliminate 2/3 of the reason for visiting the page at all, but that's not the search engine's problem. Wonder if that third piece, the phrase, is acting as "first (and only) occurrence of word three" or as "complete search phrase", or both?

[JD_Toims]

So it may even take longer for an index to populate the different data centers than it takes to create the index in the first place.

Absolutely. It used to be if you watched the DCs enough you could see it happen.

It would go something like:

First Check: Get 1 to 10 of 1,300,000 results on *most* DCs
[Normal results people are used to seeing.]

Second Check: Get 1 to 10 of 1,800,000 results on *half* of the DCs
[Different result sets between the DCs with differences in page count.]

Third Check: Get 1 to 10 of 1,800,000 results on *most* DCs
[New result set based on algo refinement and page/site addition.]

Then if you kept watching you could see things "cycling through refinements" in basically the same way -- We can't see it any more due to the load-balancing changes, but I don't have much question "essentially the same thing" is still happening.

But in the course of checking I noticed another nice detail.

Search for:
wordone wordtwo wordthree
without quotation marks. Result shows (matching the order of items on the page):

first-occurrence-of-word-one in context ... first-occurrence-of-word-two in context ... "wordone wordtwo wordthree" phrase

That's interesting. Thanks for noticing and sharing! -- I haven't noticed it personally, but I'll start looking for it now to see if I run into anything similar.

[g1smd]

You can still look at result sets on individual IPs.

You just can't type the query into the search form on an IP, and hope to stay on the same IP.

[JD_Toims]

You can still look at result sets on individual IPs.

You can, but you really can't know what DC you're actually hitting even by IP since they changed their internal load-balancing [AFAIK] -- If you've heard or know something different, please, let us know so I can look into it -- I'd love to know I'm hitting different DCs by making queries to different Google IPs.