Google doesn't often lower the rankings of a page just because it has adsense, unless it's an obvious MFA site of course, but sprinkle a few affiliate links into a site and they very well may be treated as poison by search.
I have added adsense to a page and watched as it maintained its rankings/traffic from search and then replaced the adsense with a single affiliate offer and watched it be removed from its top rankings(-200) and receive no traffic. Removing the affiliate offer in favor of adsense resulted in the rankings returning, 5 spots lower and 5 months later, and some Google traffic returning as well.
Google treats pages with affiliate offers much more strictly, apparently. If all of your traffic comes from search engines, especially Google, be very light handed with affiliate offers.