You can talk about nofollow all you want, but essentially what you're arguing is that your page was demoted by Penguin because of trackbacks. I still say that defies common sense. You say that you can't find any other explanation -- Maybe that's because there definitely are some major flaws in Penguin which are causing false positives and undeserved demotions to a lot of sites without any apparent reason.
Edit -- Another thought -- By taking the page down, you eliminated links that you originally gave to other sites, hopefully because you thought they deserved them. In my opinion, what you should do is put the page back up, restore the old content, and forget about Google.