|42% Drop in Estimated vs. Finalized Earnings - March Madness|
Estimated vs. Finalized Earnings in March 2014
I have been an AdSense publisher since 2003 (active since 2006). I have been pleased overall with the program. They always pay on time and the amount listed. I like that they don't show any malicious ads to my site's visitors. They are the best program currently for my site by a large margin. They have been nothing but dependable. I used their e-mail form to try to get an explanation on my situation. Their cut and paste response telling me to investigate possible sources of invalid activity in the future was less than satisfactory.
Here's my story. I apologize for the length of the post, but I feel like it is necessary to give as much information as possible in my case. I was shocked to see my EFT for March was 42% less than the estimated earnings. I had my two biggest revenue days, three of my top five number of clicks days, my three highest CTR% days, and my highest RPM day of all time. The madness all started on Friday, March 7. I had nearly 900 more clicks than my daily average in January and February, a CTR that was more than 2% higher than my daily average, a CPC that was more than double my daily average and my highest RPM ever on that day. Estimated earnings were over $730.
When I saw the staggering numbers developing on that day, I checked my site to investigate to see what kind of ads were running that could be achieving those levels. I frequently saw ads for Quicken Loans Billion Bracket Challenge on Yahoo. I don't know if anybody saw these ads across the internet, but they would have been very clickable. Everybody fills out a bracket for March Madness and wants to win a billion dollars, right? They started accepting entries on March 3 according to their rules page. They only accepted 15 million entries and only from the U.S. According to my March stats, 90.9% of the revenue, 77.2% of the page views and 83% of the clicks I received are from the U.S. My site receives many unique visitors daily, largely from the U.S..
My number of clicks went back to normal levels of 110-120 from March 8-11. I don't recall seeing any of these ads on these days, but they still may have been running. The ads and clicks picked back up on March 12. I had around 200 more clicks than my daily average, a CTR% that was much higher than my average and a CPC that was similar to March 7. Estimated earnings were $228. Then things became even more crazy on March 13. I had over 500 more clicks than my daily average, a CTR% that was lower than March 7 but much higher than my average, but the CPC had dropped very significantly from March 7 and well below my daily average. Estimated earnings had fallen to $61 despite all those clicks. What happened here? Were these ads so successful that they were now getting all those clicks for peanuts? This was days before the bracket had even been officially announced. The clickthrough rate of under 3% on these 3 days wasn't out of line. I actually thought they would be much higher. How many billion dollar contests do you see?
These March numbers are all still listed in my reports. There was one $.30 deduction for invalid traffic. I have read about the so-called "click bombing" on various forums. I have seen totals on a handful of mornings which were obviously inflated. These have always been corrected later in the day. These numbers in March were never adjusted down and corrected. I have to assume that they were correct. I simply wasn't paid for the many clicks delivered. I went back and compared my estimated vs. finalized earnings all time. It's always been very close. I have never seen such a wide discrepancy. There are very small monthly deductions (usually under $1-$3) for invalid traffic. My site has no history of delivering a large amount of invalid clicks. My site has always been in compliance.
I read these terms on one of Google's pages:
"Also note, as outlined in the Terms and Conditions, AdSense reserves the right to adjust a publisher’s earnings in the case that an advertiser who advertised on their site defaults on payment. Historically, however, the impact that this has had on publishers’ finalized earnings compared to his estimated earnings is low."
Is this a case of the advertiser defaulting on payment? That seems very unlikely. I read that this particular company spent $21.1 million on AdWords in 2011. Did they receive so many tens of thousands of clicks on this likely highly successful campaign that their funds quickly became exhausted? Do they pay Google only for the clicks they want and deem valid? Did they want a much lower CPC than they were orignally paying in the early days of the campaign?
A 42% difference between estimated vs. finalized earnings and a $900+ reduction doesn't seem like a regular occurence to me. It's not a low impact. They basically wiped out two complete days of my earnings. I don't consider myself a big-time publisher. I was depending on receiving close to the full amount to pay bills. Imagine my excitement when I saw the amounts on those two all time high days and the disappointment when I saw the EFT amount.
I have read similiar stories on forums about this happening in March (and in previous months). It can't all be invalid clicks. Aren't there any successful ad campaigns that receive actual clicks these days? When so many publishers had similar large daily spikes and then finalized drops in March, I believe Google should give an official explanation of what exactly happened here. This particular campaign had to be more successful than the sponsoring company was expecting. I only want to be paid for the legitimate estimated 1500-2000 clicks sent to this site. It's not even about the money. It's the principle of being paid for valid clicks and traffic sent. Maybe Warren Buffett can cut me a check since nobody won the billion.
Many of us have been in the same situation. I had huge takebacks at the end of January and February - bigger than yours. Since my sites are seasonal and those months are way off season, I knew something was wrong as soon as I saw the inflated click totals - some of which came out immediately, and some of which didn't come out till the end of the month. Multiple contacts with AdSense support were no help at all, and eventually they stopped answering altogether.
Yea, it can all be invalid clicks. I actually caught some in the act (see this post [webmasterworld.com...] and was able to block some of the ad-clicking bots that were coming from various cloud services. Not all, but most. For now.
If you're not accustomed to poring over your logs with a fine tooth comb, it might surprise you to learn that the level of invalid activity, nasty bots, scrapers, and garbage out there is beyond belief. And they're getting more sophisticated all the time - even Google doesn't seem able to keep up with it.
So yea. You got bit. If you want to continue with AdSense, you probably learn to read your log files and look for bot activity. I also found that while my AdSense Dashboard reports were way way off (and remain way off, even though the finalized earnings are way down) my AdSense as reported in Analytics was pretty close to spot on. So if you have your AdSense and Analytics connected, it might help if you just rely on the Analytics report. That's what I do now.
Google's not going to help you with this. They did just buy a company a couple months ago that has to do with click fraud detection, but I have no idea if they have that technology in place yet. My April numbers are a lot more reasonable, so I hope so.
I really don't understand how it's MY responsibility to monitor traffic and block IP addresses. I don't see why it's unnecessary for me to spend so much of my time going through logs and whatnot with a fine tooth comb. I have enough to deal with already. I should just be able to place the ad code and leave it as is. That's how it has always been for me in the past as a publisher.
It's THEIR ad code. Do any other advertising agencies put everything on you to monitor your traffic for any potential sources of invalid activity? I can't think of any offhand.
Much of my traffic comes from GOOGLE. If they don't seem able to keep up with all the invalid activity, nasty bots, scrapers and garbage out there - why should it be expected of me? Apparently their filtering system is 100% accurate when it comes to detecting potential invalid clicks. There seems to be a lot talk that they can do no wrong.
When multiple publishers are posting about the same reductions in March, something is not right and needs to be closely examined. This will just keep happening to more and more publishers unless others start speaking out against it.
FACT: It's your website, not Google's.
If a bot comes to your website and clicks on an ad. G does its part by removing the invalid click and revenue once it catches it. If the roller coaster effect this creates for you is unpleasant, stop the bots from accessing your website. BTW, the amount of server resources bots can consume is apparently unlimited. They can bring your site down, or to a crawl, messing up the user experience for real people. They can cause havoc with Adsense and, if you're paying for extra bandwidth or memory according to usage, they could be costing you money. Also, bots may in some indirect way find you via the SERPS but the fact is that they share information and so don't need the SERPS to get to you. Where Google Search and Google Adsense are concerned, it's also helpful to remember that they are different services and don't answer to one another for the most part. They're really unrelated where this topic is concerned.
Keeping your site and the ecosystem around it healthy is of course your responsibility - whose else would it be?
The bots are just a cost of doing business in 2014. Don't blame Google (well, at least not for all of it - they do need to fix their freakin' reporting) blame the miscreants. I blocked four IP ranges from Amazon AWS and at least 60% of my invalid activity went away immediately.
There is a HUGE underground economy in click fraud, on the advertiser and publisher side. Ginormous. You can't possibly overestimate it. Unfortunately it looks like it hit you too.
You can just set it and forget it - the fact that it's so prevalent is actually a good thing for you, because otherwise Google might decide your site "presents a significant risk for advertisers" and ban it or you. They can do that, even if you've done nothing wrong, because it's in the TOS that you agreed to, and as far as Google is concerned the advertisers must be protected at all costs. But because the invalid activity is so high now, ironically they're much less likely to single you out.
But it means roller coaster reporting, and you can't count on anything until it actually hits your bank. That's why I stopped using AdSense numbers and started using Analytics numbers. It's a PITA because I have a lot of sites, and have to look each one up individually, but at least I have a pretty good idea of what's coming in.
AdSense is what it is, and nowadays, that's what it is.
The bottom line is that I could do everything humanly possible to try to reduce sources of invalid activity, but there's still always going to be the chance of being hit by some. Google is always going to take out what they see fit as invalid traffic. Their #1 priority is always going to be protect their advertisers. If Google doesn't get paid, you don't get paid.
Read the official AdSense forums. I counted at least three other publishers who were reduced a similar $600-800 amount in March. One woman's story was eerily similar to mine. She had a $700+ day on March 14, but it was completely wiped out of her earnings. I probably wouldn't have said anything myself if I hadn't seen this was happening to other publishers.
Invalid click revenue is not revenue (it's a mirage in your reports) so getting "wiped of her earnings" is a misnomer. They were never earnings to begin with. It doesn't take "everything humanly possible" to get a decent handle on this problem currently. One thing's for sure, nefarious activities on the Internet are at an all-time high and it's probably only going to get worse. Learning how to keep up with the constant assault on your interests is more critical now than ever. This recent click bombing episode was a great wake-up call for some. The problem is pervasive so ignore it or blame it on someone else at your own peril. You're right, it's a never-ending task but even small steps can go a long way to protecting yourself.
If you want to find the problem, you need to start segmenting your traffic. And you don't mention whether or not your AdSense account is linked to your Analytics account - you really need that in order to go in depth (unless you want to just read raw log files) Analytics will give you information that AdSense won't (even earnings information)
1. Were your March metrics (before the takeback) bigger than usual, or about normal for time of year and amount of traffic?
2. As far as I can tell, most if not all of the invalid activity comes from Direct traffic, not organic traffic and not referral traffic. So start by segmenting out your direct traffic. Use your stats or analytics program to analyze what that direct traffic is doing - does it come from the countries you'd expect it to, or are there a lot of anomalies? (since my traffic is targeted to specific states, a lot of visits from Europe or Asia immediately sends up a red flag) How many pages per visit, and does it come to the same pages over and over, or dig deeper into your sites? Most of these bots seem to visit the same two to five pages over and over, and the average time on site is less than 7 seconds.
3. This is where having the AdSense/Analytics hookup is useful. First of all, compare your Analytics numbers to your AdSense numbers. In my case, Analytics is closer to real. Then you can burrow down by acquisition source to see if your earnings are coming from search traffic or direct traffic, and which pages earn the most. You can even look by provider - which is how I found that Amazon AWS (and a couple other providers, like XLHost) were problematic.
Once you dig down that far, anomalies may start to jump out.
No, you'll never get it all. Click fraud is bigger than you, and I'm pretty sure by now it's bigger than Google. That's why they bought the click fraud detection technology recently.
It's happened to way more people than the ones in the Google forum. And Google is not going to make any statements or acknowledgements at least until they have a better handle on it, for fear of tipping their hand on what they're doing - the people behind all the invalid activity are pretty smart, too, after all.
Of course, if it's too much to deal with, you can start thinking about other sources of monetization, like direct ads. It's a lot more work, of course, but you have a lot more control. AdSense has a lot going for it - ease of implementation, dependable payment schedules, big network of advertisers - but in exchange for all that, you have to give up a lot of control. I don't think Google wants or needs to change that business model; it's working for them.
(We all bitch about Google here, sure, but we're also about finding solutions and workarounds. MY bottom line is that I have bills to pay, and so I need to find a way to work with the AdSense system, or else find another system. Spending time on anything else doesn't pay the bills)
I appreciate the responses. They are interesting and helpful to read going forward. I use AdSense as my primary advertising network for all of the reasons you listed. There is a big trade-off compared to the other major advertising networks.
There seems to be no manual review in place. Anything that looks abnormally high compared to your recent averages seems to get flagged by their filtering system. The three days where my numbers were much higher than my daily averages weren't out of the realm of possibility. There are some highly successful ad campaigns being run on AdSense. Real people do click on ads that are targeted to them and look appealing to them. Maybe not like they used to, but the clicks are still there. That's the very frustrating part.
Google has never been terribly interested in anything that doesn't scale. Manual reviews don't really scale, at least not to the extent that they'd have to do them to battle the increased activity. There's what - a couple million publishers? No, if they have to use resources, they'll probably use 'em on the advertiser side first.