I haven't seen anything that would indicate that affiliate links hurt SE rankings. If a site has no content or load of dup content, it will get dumped eventually, affiliate links or not.
Thinking in terms of advertiser/publisher is the wrong approach. If CJ wants to be an advertising network then this may be the right direction to go. Affiliates are not marketers, they are sales people or organizations.
Banners are the standard brochures sales people may hand out to prospective customers but the content is the sales pitch. If a sales person had to walk around with a tape recorder from corporate management that repeated the same thing to every prospect, they wouldn't sell much of anything.