This is the point. If its been something that allowed a site to manipulate its rankings in the past then its for sure google will look at it at some future date to try and filter for it. It seems the only effective way to stay out of the filter these days is to have a decent trustrank score. If you dont have that then you have to be very careful with everything onsite. To make it simple i would say you can do a lot of one thing but not a lot of a lot of things. Your example is the exact type of thing i have been looking at. For me i would say you have to back of little by little and see if it helps. But this is still guesswork as to whether it really is your problem.