It sounds to me like your site's pages only fell slightly (1-3 places) when Penguin originally hit it. That's what also mostly happened on my two Penguinized sites. I think these slight drops were due to a devaluation of low-quaility backlinks.
But some sites were hit much harder, in a way that resembles an active penalty.
I think there are two aspects to Penguin:
1. A devaluation of low-quality backlinks that cause small ranking losses.
2. An active penalty applied to some sites in addition to the backlink devaluation.