We did some hoop jumping after the Mayday changes last year and found that it was nothing but an exercise of blind faith. Change servers, reduce clutter, re-organize content, etc etc. When it was all said and done, our site recovered... but we really had no clue why.
We weren't really effected in either way from Panda... but we already decided that if we were, we would do nothing but find other streams for the lost traffic.
Incidentally, that was also part of our plan during the Mayday update last year, and we feel that those added streams of local marketing and demand generation have been the best investment of our time and money.