I usually think of this type of situation as a bit of a 'hot news' check...
If you've got 1 Mil+ pages for a result set, get a new 'quality content' page and want to know if it's 'news' or not, you could insert it in some results (maybe only after an initial similar search and only one set of data-centers to keep the 'click fakers' from skewing the determination too much?) and see how the clicks compare to surrounding results.
If people start skipping over the 'usual answers' and clicking on the new result at 'high' rate, it could be an indication you've got something 'fresh and new' people are really looking for, so you could 'increase the visibility' and if not, then page could be returned to its 'natural position' to 'work' it's way in to the top results like all the rest.
Think of it like split testing... New page, on-topic, etc, etc, passes all the 'sniff tests', could be important... Page gets inserted to a small portion of the searches and visitor behavior aids in the determination of 'hot story' or 'regular result', and it's very difficult to 'fake a click' because it only shows 'intermittently' on Google.Com and where it shows could even be GEO Targeted and the targeting expanded / contracted based on behavior. (Just as one example of how you could do it.)
IMO, regardless of some member's views regarding clicks, there's quite a bit of information you can get from the data, especially before you make the resultset available to everyone... Once it's on Google.Com consistently it's much easier to fake clicks, but you can't fake a click on what you can't find in the results, so by limiting the availability, delaying the page from showing in the site: search and moving it in and out of results, you can 'gauge responses' on a smaller 'random' (for lack of a better word) scale, which gives you a much more reliable comparative pattern of behavior prior to making it easy for anything to be faked.
Personally, when I put new pages up I'm usually happiest when I can initially find them on Page 3 at a few specific datacenters... That's where (how) most of the long-term, stable, well ranking results I have started.
The disappearance you're experiencing could easily just be the datacenter you are hitting too. I know one site I recently made additions to has pages I added about a week ago showing in the SERPs on certain DCs, but the only way you can find them using the site command is with site:example.com/directory/ and they do not show on Google.com for me, yet...
I've been watching almost the exact same repetition for years... Page 3 on some DCs. If they follow the 'usual pattern' I've seen around the time they move up to Page 2 on those DCs they'll start to show on Google.Com (Page 3) and there will be more pages indexed on certain data-centers, depending on the propagation of information to individual DCs by Google.