Links on pages, thin content (whatever that means) etc is on most sites so Google cannot penalize for them, as much as they'd want to. Penalizing the 'wrong' sites costs Google credibility, so pick a 'good' site and try to achieve what they have. I have wasted way too much time listening to Google's supposed advice and focusing on stupid stuff, as if a spelling mistake can justify a 400 rank drop.
IMO, most pandalized sites are not supposed to come back on Google, almost no matter what you do. That's why Panda is a sitewide penalty, not a page by page. It's not personal, just business.
Edit: I agree with Muslina, visibility doesn't have to correspond to loss of traffic. You could be visible but barely get any traffic from those keywords so the loss is not much, if that visibility is lost.