I have also been racking my brain to figure out what to do.
First, I looked at all the nofollows on my site which were affiliate links. I know Google does not like nofollows.
The odd thing is one of the sites listed on the "winners" list has bunch of nofollows on them. Site seem a wreck with no consistently throughout. Some have the entire navbar on the left nofollowed and sometimes it is dofollow. snopes[dot]com . Look under "Odd news".
I have also moved my Adsense block from the sweet spot to my sidebar and thinking about putting up a leader board.
I am not rewriting any content because I strongly feel that it is all relevant and since it is all original, I stand by it.