Forum Moderators: martinibuster
I am running a click tracker, one of the tools that may be run next to Adsense without violating TOS. Great stuff when I was new, but today rarely used. For me, the tool got less useful with the introduction of IE7 first (which blocked the status bar displaying the target URL of a click) and even less useful when Google decided to not show the target URL as part of their new ad layout. Since then I am "flying blind" as to which target URLs are being clicked. But I kept running the tool anyway in order to see at least where the clicks were generated, and which formats/colors work best.
Now, twice a year I am comparing the official Adsense stats with my inofficial click tracker stats. I just dump the data from both sources into an Excel spreadsheet and compare the values.
With very interesting results.
Adblock views
For some strange reason, the tool is overreporting views. Always has. It claims more ads have been viewed than Adsense tells me. The level has been constantly between 20% and 30% deviation. Probably bots, I thought. And frankly, I do not care that much about views. I am more interested in the accuracy of click recording...
Clicks
Here it gets messy. The tool has been virtually always underreporting the clicks. When I started, the deviation was as much as 10% (i.e. Google recorded 10% more clicks than the tool), but over time it got better until in September 2006 both values were about the same.
Then, suddenly, Adsense begun recording more clicks again, and starting in December 2006, the tool just captured about 65% of the clicks Google recorded. I attributed this to the fact that IE7 was introduced and the tool was an old version, so it could not capture all the clicks any longer. I updated the tool in March 2007, and things were back to normal, with the tool again slightly underreporting.
Early-May to Mid-June saw another harsh discrepancy between Google and the tool, again Google recording more clicks than the tool. This time, however, I did not update the tool. Things went back to normal Mid-June without me doing anything.
(I can hear your yawning, but just hold on, now it's getting interesting.)
At the end of August 2007, the tool suddenly began overreporting clicks! Yep, that's true: today the tool captures MORE clicks than Google tells me. In the past this has happened before on single days, but the 30-day-average was always underreporting clicks, since end of August I am seeing the opposite: The tool is recording/displaying MORE clicks than Google. Please note that I did not update the tool, so all of a sudden, clicks are not being accounted for. On some days up to 30% of the clicks are not being displayed by Google.
This began end of August, and guess what? My eCPM for September was really really bad. Just co-incidence?
Of course, it could be the tools' fault. Yet I wonder what might have changed that a historically underreporting tool suddenly and without me doing anything becomes an overreporting tool.
The only explanation might be that someone has been stealing the page (removing the Adsense code, replacing it with his code) and has kept the tracking code on the pages. But the tool does track this as well, and I have been seeing about 0.3% of the clicks coming from such pages. So, this does not explain the 5-7% underreporting of Google.
My questions to YOU
1) If you are running a click tracker, are you seeing the same or similar problems?
2) What might be the reason for such a strange behaviour? (I'm still unwilling to believe that Google has just decided to 'take away' clicks.)
On June 1st, there was a WW thread on "3rd Party AdSense Click Trackers Disabled" [webmasterworld.com...]
Without tracking the clicks, one just has to trust Google's word, which is not what you'd want in a financial arrangement such as Adsense.
In addition, it seems fishy that all the oddities with adsense seem to have started after the disabling of 3rd party click trackers.
As WW member frakilk put it on June 1st:
I'd say that this is a deliberate change. Something's a-brewing at the Plex.
If there are still click trackers that seem to work, a list of such tools would be of great value at this point in time.
ADDENDUM: I don't use a click tracker, but I do use log-analysis software, and I've always gotten slightly different numbers from different log analyzers. I get the impression that counting is easy, but deciding what to count is where the heavy lifting comes in.
Google has always reserved the right to discard clicks that don't meet its criteria as "valid clicks." For example, if a user clicked on an ad several times in rapid succession, Google might assume that the second and third clicks were errors and discard those clicks.
I was under the impression that Google counts/displays all the clicks, while the actual valuation of the clicks happens at a later stage through SmartPricing.
I always wondered why in September my eCPM went suddenly and significantly down. Now that I know that the number of clicks Google reported were down as well (compared to the click tracker), it suddenly makes perfect sense. To me, that is.
But how can I know? How can we know? Too bad that Google is so bad at communications (which is somehow funny, in a sad way, for an Internet company)...
Before the maintenance I was getting the expected number of clicks (low) but since then my normally low ecpm and ctr has been halved as well as the number of reported clicks. Money is in the toilet, to put it delicately, and I'm thinking of pulling the handle on adsense and flush them away if things don't start looking up.
..All that as well as putting my site for it's major keyword on page 6 of the serps after being on page one for ten years! I have always been white hat, no tricks and it is a good informative content filled site.
I think I am ready to see some changes even if I have to make them.
Ann
If you want to have a discussion about possible variations in reporting, why not post the exact method the click tracking script you've implemented uses to infer whether or not a click has occurred? Then we can have a specific, practical discussion about the click tracking accuracy and possible sources of error.
Google keeps finding ways to take power away from us and give it to itself. It is not becoming more transparent; it is becoming less transparent.
Has it ever explained why it started deliberately blocking 3rd party tracking software to hide the stats? Or why it started hiding the target URL?
Wasn't it a blatant decision to keep itself from being held accountable?
Payout rates are non-negotiable and 100% hidden. Smart pricing is hidden.
I get an increasingly uncomfortable feeling being tied to a company that wants to keep you more and more in the dark. And Google increasingly uses cut-and-paste boiler-plate (evasive) responses to direct email questions.
Keeping the programming glitch of the last week hidden also doesn't sit right. It's just the latest in the continued erosion of trust.
I'll stick with Adsense for now but I'm not buying Google stock.
p/g
P.S. Has anyone ever noticed how AdWords users have long threads on new features they'd like to see (they're now up to about Wish List 10.0), but you almost never see one here?
Click trackers CANNOT track clicks directly. They can only make educated guesses about what has transpired in a user session. They can still provide roughly accurate numbers, but they're merely making educated guesses. Unfortunately asking "Why doesn't my script's educated guess match Google's exact numbers?" doesn't have the same flame-baiting impact as "Is AdSense underreporting?"
why not post the exact method the click tracking script you've implemented uses to infer whether or not a click has occurred?
Great idea.
I was under the impression that Google counts/displays all the clicks, while the actual valuation of the clicks happens at a later stage through SmartPricing.
Google displays all clicks that it records as clicks initially. That doesn't mean it displays all clicks.
Also, smart pricing is a system of advertiser discounts from the nominal bid price. It isn't related to click tracking or click validity; it's about Google's estimation of whether a given click (or a given publisher's clicks) will convert.
Google displays all clicks that it records as clicks initially. That doesn't mean it displays all clicks.
This sounds like you have some background information others here (including me) seem to be lacking.
Anyway, your statement means to me that Google does not record all the clicks as clicks, right? In other words: there seem to be some criteria unknown to the wider public that make a click "disappear". Care to shed some light on this?
Click trackers CANNOT track clicks directly. They can only make educated guesses about what has transpired in a user session. They can still provide roughly accurate numbers, but they're merely making educated guesses.
I agree. Well, I did agree for a long time.
As I explained in the first post, the tool was overreporting page views (I don't care) while it was underreporting clicks. So its "educated guess" was not as good as Google's recording of the same clicks. For a very long time, Google was catching a higher number of clicks (like, "all clicks") and displayed these clicks to me. The tool was reporting less clicks (like, "not all clicks"). That was OK for me. I attributed this to the fact that the tool was not capturing all the clicks, for whatever reason.
But sometime early September, this behaviour changed without me having changed the tool. And so, in September 2007, the tool (using the exact same method that previously caused to underreport) was now suddenly overreporting, i.e. showing more clicks than Google.
And guess what? This behaviour again has changed about two weeks ago. Now the tool and Google seem to be aligned again, roughly reporting the same number of clicks with a slight tendency of the tool to underreport.
I am still willing to believe that no "evil" intentions had caused this problem. It might have been a glitch, it might have been an update to some code that did not turn out well, it might have been an intended change to, say, capture invalid clicks. But it might also have been a money grab to beef up the Q3 earnings. (Again, I still do not believe that it was a money grab. Google could just turn the SmartPricing knob to achieve this. Yet, it could have been a money grab.)
As always, the lack transparency (and communication) is the major problem. My trust in Google is seriously damaged. (Of course, one might argue, that if Google does not tell us about a problem, then there was no problem, so there is no need to tell us about it. *grin*)
Could someone open a window, please. It smells fishy.
Nahhh...never happen
Ann
Since they are trying to take over the sites by forcing everyone to their will
That's increasingly how I feel about Google. I know there are other advertising networks and other ways to make cash online out there but for small publishers it's pretty much the best they can get. I think Google knows this all too well and is using it to their advantage.
AdSense is so ubiquitous it's scary. It has got to the point that whenever I visit a new site I start to look around for the obligatory AdSense block. Usually I'll find it.
My 2008 New Year's Resolution: diversification!
Since they are trying to take over the sites by forcing everyone to their will, other engines be damned, makes me wonder how long before you have to sign over your sites to Google in order to show thier ads?
Huh? How is Google trying to "take over the sites"?
Let's not let venting get in the way of common sense. Fact is, AdSense is "ubiquitous" because (as others have said) it's the best deal that most small publishers have going for them. It's also a useful tool for not-so-small publishers, because it complements other revenue streams (such as display advertising and affiliate links).
If AdSense stops being "the best deal going" for small publishers and a useful tool for larger publishers, publishers will make other choices. Google knows that, and so do current and potential competitors. (Remember when Amazon.com link boxes were all over the Web? How many of those do you see today?)
This sounds like you have some background information others here (including me) seem to be lacking.
There's plenty of background information around, but you need to look for it. You might start by reading Dr. Alexander Tuzhilin's "The Lane's Gifts v. Google" study [googleblog.blogspot.com] of Google's click-fraud detection methods. (It's in Adobe PDF format.)
EFV,
congrats for 10,001 postings! That's 10,001 times knowledge and wisdom from a seasoned web veteran who actually has "made it" in terms of popularity (of the web site) and independence (financially).
We may not agree upon Google, Google's actions, and Google's perception in the real world, but that's what makes discussion forums interesting and worthwhile to visit. Well done!
1. "Something" has happened on or around 5th September. To me it looks like Google pushed an update that was (intentionally or unintentionally) "too much" for the affected publisher base, or maybe even for their product managers. From my data set, I see a correction on or around 14th October. From then on, the bold underreporting by Google has been replaced by a slight underreporting by Google.
2. This past week has been very close to the worst week in 2007. My worst week fell into the period 5/9-14/10 which is no surprise to me. I know that a part of what I am seeing is seasonal. To me, this looks, like Google is testing "low waters" - how low can we go before this site removes ads?
3. I am certain that something else is going on. Ad quality seems to have improved, at least for the markets I am monitoring (US, UK, Canada). The same advertisers as always. CPM in October has gone back to the level of August (so, September was clearly way way off). EPC has improved. CTR has improved. Yet, I am seeing the same revenue per day as in September.
I feel like Google has assigned a monetary value to my website, regardless of traffic volume or click quality.
I feel like Google has assigned a monetary value to my website, regardless of traffic volume or click quality.
This seems to be the feeling some people get (myself included). If we assume it's really the case, my question is: to what extent are we able to change this "value"? Is it a completely algorithmic decision or human review is necessary to re-evaluate our sites?
But frankly, if this proves to be the case, I am going to be very disappointed. This certainly gives you the feeling of hopelessness - like, regardless of how hard you work, you can't substantially increase your income.
P.S. Is anybody here familiar with "game theory"? I think they use some of its techniques for determining "optimal" AdWords pricing and I suspect similar algorithms are used for AdSense. You mentioned the "how low can we go?" approach - I think it's actually the point of the theory: i.e. finding the exact "breaking point" and trying to push you as close to it as possible. Obviously, the "game" is played in opposite directions with advertisers and publishers.
[webmasterworld.com...]
But, I also feel that this ceiling is slowly raised or maybe also decreased (if we see the recent posts by many publishers)
At times, I see the adsense team 'experimenting' with this ceiling and my earnings go up 30 - 40% ( mainly on a/c of EPC increase).
Currently seems to be on an increase.
There is definitely something more than smart pricing / advertisers budgets etc going on.. We can only guess..
