walkman - 4:22 pm on Oct 17, 2011 (gmt 0)

is now occupied with 4-5 solid sites that usually occupy the space dispersed throughout the top 10 results, along with 3-4 spammy sites and 1 that isn't even on topic

If they compare this data with all the other sites that have previously occupied that SERP spot they'll be able to make a better estimate on who deserves to be there. In the end then they'll have, according to their data, the best possible sites in those spots. They may need to continually and occasionally cycle in new sites though just to keep the front runners on their toes as far as providing valuable user experience. Since our site has decent time on site, bounce rate, etc and by comparing it to the major sites that have been there for years, we know we provide at least an equal user experience. So maybe it's just a matter of time before we bounce back.

I think you may be unto something. Just consult these guidelines "More guidance on building high-quality sites [googlewebmastercentral.blogspot.com...] " and Google will make sure the best possible sites go on top. In many niches, Google has removed a few spam sites that ranked on top 8 months later so there's hope.