i use alot of affiliate links and from my perspective the bigger concern is not seo but conversion rates. when people scroll over a cj affiliate link, the url can look ugly & untrustworthy. to make a better looking url when users scroll over the link i first send them to a subpage on my site. that page then automatically 301 redirects to the affiliate page. i also use robots.txt to block that subpage to reduce the affiliate link signal. when i made this change my conversion rate went up quite a bit.
deep down i still think google knows there are affiliate links on my page. i provide a good user experience with a high time on site and low bounce rate, so I am seen as ok. imho most affiliates that get penalized are because their websites are horrible and a terrible user experience.