Advertisers can contact publishers directly now. Most don't, for several reasons:
1) It's seldom cost-effective to deal directly with publishers unless the publishers can offer a high volume of impressions.
2) Only in very rare cases would a publisher be able to offer the degree of contextual targeting that's available through AdWords/AdSense.
3) AdWords/AdSense lets advertisers spread their money (and their risk) around by aggregating impressions and click referrals from many sites.
As for another poster's suggestion that domain blocking for advertisers is impractical because there are too many publishers, that's like saying that domain blocking for publishers is impractical because there are too many advertisers.
IMHO, AdSense ads with more advertiser controls will come, either because Google recognizes the need or because the next generation of AdSense competitors forces Google to protect its market share. As I've said before, AdSense is a first-generation product, and one would have to be naive to assume that Google will never offer product extensions that can attract a greater variety of advertisers and offer higher profit margins.