It isn't entirely true that CPM is falling for all publishers. Actually, it is rising for some and as the economy recovers, it is expected to go up.
Just remember, today's AS publisher with rising CPM is tomorrow's AS publisher with falling CPM. Not all sites with AS issues are junk sites, and not all of them are run by people who don't know what they're doing.
Be glad your site is doing well, but accept the fact that it COULD change for no discernable reason whatsoever tomorrow. It wouldn't be the first time.
That's not to say there aren't issues with some of the sites involved, but it's not the case with all of them. Something else is happening.