I am sorry to hear of your troubles coachm, and a lot of what you said should be taken note of.
However, I am specifically replying to the title you gave this post CTR, visitor rejection of 'Google' and what I am doing about it - rather than other issues you pointed out, like a decrease in earnings. I might be slightly at odds with you in some respects because it sounds as if we have different types of business plans and different website properties, (but I may be wrong on that).
My target audience is the Western English speaking world, specifically UK, USA and all other English speaking countries tied into that demographic ... what does that matter? Well, it concerns CTR ... yes my visitor numbers continue to climb from my target audience, (as does Adsense income from them) but visitor numbers are climbing in 'real terms' faster from the rest of the world (TROTW) who do not have English as their first language, and the reason is simple.
Just as in the 1990s computer usage exploded in the Developed Countries, specifically the UK and USA, it is now exploding in TROTW, including less developed countries. These countries are now on line and they are searching for things ... admittedly this is still found in the richer sections of these societies, but the trickle down effect is also happening too.
Also computer usage in the relatively poorer Ex Eastern Block Countries is 'booming', and although in many of these countries English is their second language they are very, very, proficient in it.
For better or worse, 'because we got there first', the most important and most diverse information on the internet (at the moment) is written in the English Language, therefore people all over the world are seeking this information using English and English centric search engines like Google to find it. In the EU Google has over an 80% share of the search engine market, including in non-English speaking countries. Also other search engines are crawling my site looking for information, ones which are not English Language Centric.
What does all this mean? Google et. al. (but particularly Google) is sending me more and more traffic from less developed nations whose second language is English, (seen in my sites analytics results) they are being sent by long tail keyword searches which match pages on my site. So visitor numbers are up, but CTR on Ads is down, and there is a correlation there.
I am not sure if foreign language visitors have found the information they are seeking or not on my site, (who knows) but they are coming in increasing numbers, and while this is ok, (good information is still good information - if you can read it) I don't target them because my writing style and niche information is heavily grounded and based around the assumption that English is the first language in the country of the visitor - and all the nuances and cultural sensibilities and economic development which that entails.
Note: I also would happily agree that visitors from more developed countries like Germany or France might not find my site as interesting as one similar based in their own language written by native speakers.
Ok, lets get to the point ...
Google Adsense just released numbers in the Adsense analytics section which tells me how many countries my (their) Ads are showing in and the CTR for those countries - this tallies very closely with my Analytics Results of numbers of visitors by country.
It also matches perfectly with my assumptions (previous to getting this data) that CTR from Adsense is non existent, or almost non existent, in countries whose first language is not English - and these Ad numbers are showing thousands of impressions from over 179 countries around the world. I knew this, or rather 'felt' that this was happening, but I could never back it up by hard data ... until now.
So what have I done? Basically my hesitancy has been removed (on the 6 weeks data available - from March 22nd - which is enough to show me the trend that I 'felt' was happening, but you might want to wait for more data before making a decision) I have acted. The last four days I have used my third-part Adserver (DFP) to only show Adsense in 17 chosen countries (which will be 16 if Australia don't get their finger out!).
Based on this new information, and the blocking of poorly performing nations, my CTR is now through the roof, while my Ad impressions have plummeted, but obviously the money earned is still the same - all pretty simple mathematics, right? Oh and if you were wondering the data in the Adsense performance tab, by 'country', shows that the targeting is working, only the same 17 countries now show up each day.
But will this increased CTR in Adsense, which now looks like pre 2006 levels, help? I am making no more money with these new figures, but will it help me to make more money down the line? ... or will I get smart-priced because of a suddenly 'inflated' and 'healthy' click through rate? All good questions, ones I am looking forward to finding out! But that is the point, I am doing something about my CTR, so perhaps don't look at CTR as a problem? Focus on the 'loss' of earnings?
Are you making the same amount of money as you were 5 years ago, with inflation making the money go less far? Is your CTR in your target audience countries actually 'healthy' but it is being 'diluted' by an increasing number of visitors from non-target countries who are not clicking on Adsense ads? etc ... have you actually stopped expanding / stagnated / decreased in your target market area, and an actual expansion in visitor numbers is a 'false' positive, because these extra visitors are less focussed on what you have to offer?
Given all the data set, stripped of the non-targeted audience, my Adsense CTR is ok, while the visitor numbers continue to rise from my targeted countries, so overall, long term, I am making more money. And looking at the positives I now have the opportunity to sell Ad space to anyone interested in targeting my other visitors from TROTW. Or, I can wait until Google can offer localised and specialist Ads to these regions which would entice them to click on them, something the CTR suggests that they cannot do at the moment.
Decreasing CTR is something that I do not blame Google for, they are trying to show ads to non-targeted visitors from these countries who are increasing daily; the coverage % of ads is very high, but exactly what type of Ads are being shown I have no idea, but what this new information shows is that they are not being clicked on. Is this surprising? Not really. At the moment most online advertisers are from the English speaking developed world, time is needed for other countries to play catch up.
Your complaint against IBA ads is a whole other kettle of fish by the way coachm
My advice? Don't use the Adsense 'set it and forget it' code, (you have no control over it) put it through a 'line item' in your Adserver and control which countries get to see it ... (now this next part might sound like gobbledygook to those who do not use an Adserver but make sure your Adsense 'line item' is not competing against Adsense activated in the 'Ad Unit' placement - if it is, when your 'line item' Adsense is set not to show in that country the 'competitive' Adsense code gets shown instead, defeating the purpose).
This way you 'may' be able to get your CTR back up to a healthy level and sell off your 'Asian' or 'South American' traffic (Ad impressions) to a company interested in cheap CPM in that region who have Ads which suit that demographic / market.
Look at expansion and contraction of your visitors only in the regions / countries which convert for you ... strip out any increase in visitor numbers over the last three years from countries which do not convert, what do these numbers show, an increase or decrease?
But first go look at the number of countries your Adsense code is being shown in, how many thousands of impressions over the last 6 weeks were 'wasted' diluting your actual CTR? This is something you can do something about, and if you use DFP as an Adserver Google is actually giving you free and very powerful tools to do something about it with ...
