I agree with your direction - but I wouldn't exactly call Panda "simple" as an entirety. In the case of any one site's demotion, maybe it is. For my site basically what it took was staking a claim to our content authorship instead of literally giving it away. But across the whole spectrum of sites being analyzed by Panda, I don't think the algorithm is at all simple. It's just that the slice of the algorithm that YOUR site gets hurt for may be.
However, I do think that gathering in-depth user data points toward an important future direction for SEO. Here's my current thinking on this. Google is learning to use this data. Bing is learning to use it. But we site owners have direct access to our visitor's browser experience. We can, potentially, do this analysis better and more completely than any search engine.
Not only that, but when we then take action to improve the visitor experience based our user data, we essentially "cut out the middle man" (the search engine). Instead we're doing something directly for our visitors and only indirectly for the search engines. Since they will see the improved user engagement, this kind of action still optimizes search engine rankings - it's still most definitely SEO.
If site owners begin to gather the kind of browser data that search engines can (did the visitor scroll the page? did they hover over a link?) then we also have a real advantage when we do A/B testing because we have some major clues about what changes have the best chance to improve things. Often that's the hardest part of A/B testing - figuring out what changes to test!
Right now, as far as I know, there are just a few paid services that will report on just some of these browser measurements. There is also a free application called boomerang.js that can give some great insight into page speed for each user - instead of some abstract testing service.