Speaking solely as a publisher:
This is a business. Like it or not, even for you webmasters with "hobby sites" - from the second you put an affiliate banner on your site to generate income - you are a business.
Every day thousands of businesses enter into agreemetns with other businesses. As publishers, we enter into agreements with another business - Google. When you enter into any agreement, you are restricted by the terms you enter into with the other party. In our case, we have agreed to enter into the terms of the agreement Google has set forth. If there is a problem with the agreement, it may be recinded or terminated according to the terms of the agreement itself. However, this is where the issue and frustration lies -- if we terminate our agreement, where do we go from there?
When Google is paying out nicely with high CPMs and we are getting a lot of traffic from the SERPs, we are happy. When CPM and traffic falls, we gett pissed. If you have been doing this for long enough, you know that there are ups and downs and you learn to ride the waves.
When traffic and CPM goes up dramatically over night, we take what comes without asking any questions. When traffic and CPM falls dramatically over night we become angry and frustrated. It is in this latter case, that Google needs to provide better communication.
Google has and will continue to make changes to improve its search results and provide a better experience to its users. I hope they continue to do so. However, as a publisher, I would like to believe that they also care about my site as well. They make lots of money off of AdSense, but without us publishers, there would be no place to put the ads (although G could create or buy its own websites and rank them high in the SERPS - but that is another discussion).
So, coming back to the discussion above about business relationships -- as business partners we deserve the courtesy to let us know when something occurs that might be affecting "abnormal changes to our relatationship" such as a 50% change in stats over night or being "banished" either by black boxing or penalization, etc. It is the lack of communication at this time that is the most frustrating to publishers. When this happens, we sometimes come to the forums to complain - sometimes to let off steam, and many times to assure ourselves that we are not alone. I would hope to beleive that individuals from Google are reading these forums, so they know that these issues do persist.
Google needs to communicate better to us at these times of stress. Unfortunately, as it has been brought up in this thread and in many other disussions, there is no real affiliate alternatives to turn to. So, we stay and continue to be frustrated. If there was another viable solution, then maybe Google would take more care to ensure that publishers do not leave. Simply, this comes down to better communication. No, we do not need to know the exact algo that goes into their search. No, we do not even know what G's cut is for an Adsense transaction. What we would like to for them to say is "Hello partners, this weekend we are making a change to algo in order to weed out [you list the type of junk sites that come up in the SERPS] sites and we are focusing on word density. If you are using particular words or phrases too many times within any page, please rethink your content." [Yes, I might be dreaming]
I am not saying that they are an uncaring corporate monster, but as a publicly traded company, their first order of business is making money. Google is a good company, and I like Adsense - I even love it when doing well (as I am for the moment). It is also the best viable solution out there for affiliates.
So, as for most of the black box issues, if it is a private Google-publisher or Google-advertiser item, let them keep it secret. For most algo changes, they can keep it secret as well. If a site's statistics are drastically changed - then a little more communication would be nice. The argument that they are so big that they should not have to worry about individual publishers should not even be brought up. If you buy a car and a tire blows when driving on the highway then you wind up in the hospital -- are you going to say that since X tire manufacaturer is so large with many millions of tires sold, that I am just one customer and have no right to sue? No you would not. If Google is going to enter into all of these hundreds of thousands of agreements, then every business relationship should be given the proper respect.
In summary - Most of the time we do not need to know what is going on inside of those black boxes. However, when the time does call for it, Google needs to provide the proper communication to all.