I haven't seen any evidence that Google hates affiliate links per se. Affiliate sites? Maybe, or maybe not. Boilerplate affiliate pages? Definitely. GoogleGuy made that very clear a while back with his comments about the need for affiliates to add value.
For what it's worth, I used to have CJ links on every page, and now I've got them on just a few pages, but I've never had any reason to believe that Google cared either way.