About a year ago, Google did run a test of multiple results that were most likely a precursor to these... and I'm suspecting they were triggered by less sophisticated testing methods than what we're now seeing in action. To sum up a range of discussion, at the time we saw these as tests of [site:domain keyword] type searches.
New multiple indented listings with meta descriptions on some searches
Sep 30, 2009
How to get 5 additional collapsed Listing in SERPs?
Oct 11, 2009
As I'd mentioned in one of these earlier discussions, "Google announces the features that stick", so I think that the feature is now here to stay in some form, perhaps to be modified over time.
I believe I have been recently seeing ongoing tests continuing with these results... in particular that the choice of additional pages seems to be rotating over time. Changing browsers, flushing cookies, pausing web history, signing in or out, etc, doesn't seem immediately to affect the choices I see returned in the serp for a query. From previous tests I've run, I'm suspecting that once a page gets in the queue, it stays there for a cycle of testing.
I unfortunately don't have access to the traffic data of the site on which I'm observing these changes, but I'm guessing that Google's goal here is to return the pages that attract the most productive traffic for certain types of searches... and that most likely this data is personalized.
Disabling personalization does affect which choices I'm given in Instant Autocomplete (aka Suggest)... or at least it does at the moment... and this may eventually affect what multiple pages appear. I think Google would like users to stay signed in because that gives them a set of reference behaviors to work with and ultimately to customize results.