I think it's easy to say a site doesn't rank because they've done something to earn a penalty, then disregard any of the other factors that might be a consideration. Your average searcher doesn't care about Google penalties, if they even know such a thing exists. All they know is someone told them about this great site about hand made blue widgets, but they couldn't remember the URL. So, they go to Google and type in "hand made blue widgets". What do they get in the SERPs? Do they get handmadebluewidgets.com? No, they get 50 or 60 directories that have scraped sentences from handmadebluewidgets.com, then 30-40 other sites that might be related that mention or link to handmadebluewidgets.com, or they get several dozen sites that mention blue, widgets, hand made, hand, etc.
That is NOT a relevant search. Especially when the site being searched for is buried at #256 for its own name. It's only a matter of time before the public notices this is happening, even now I've had people tell me they feel like they're wasting their time at Google. How long before the switch to a new SE happens?
Face it, there are some very basic SE criteria that Google is failing to deliver right now. And if Google is going to penalize sites for some unknown factor, and replace the penalized site with a hundred or so sites that MENTION that site instead, how is that relevant? Google isn't helping people to find what they want, they're throwing up detours and barriers instead.