tedster - 2:45 am on Jul 16, 2010 (gmt 0)

You might be on to something there. Remember the old days when there was a single, clearly labeled, Supplemental Index? And then it went away - with many of us suspecting that several different kinds of "supplemental" partitions had been introduced into Google's data storage?

So what if... low priority URLs are tucked away in second or third level storage somewhere and the regular site: operator doesn't access that data. But make the search more complex by asking for site:example.com/directory/ and voila! you force the look-up to go into the dark corners of the data center.