I think it was the type of keyword. Panda 1 sites ranked for the trophy one word and two word keywords.
I gained traffic in Panda 1 and got nailed by Panda 2. I was ranking for some pretty good 2-word phrases, but interestingly they are phrases that have good traffic but don't have a lot of money in them (and thus not a lot of adwords advertisers).
I bet they used adwords data to determine which keywords to look at first. The higher the adwords bid, the more likely it is to be spammed.