I wanted to run down with everyone the steps we took to identify it, in case some people are curious as to how you catch it.
Steps for investigation that we took:
1. Observed a much higher than normal traffic volume (we're talking orders of magnitude higher)
2. Looked at my AdWords account, figured out these clicks were content and not search clicks.
3. Traced it to a specific keyword via our tracking system, then dumped all session information originating from that word and the AdWords source (we use a SQL based landing page system, so it's an easy 5 second query)
4. Compared offending IPs with known open proxies. No match, and the most clicks by a single IP was 7. So there was really no patterns within the IPs that were clicking.
5. Sorted by referrer in Excel, noticed that most clicks were originating from three main AdSense pages
6. Noticed that those AdSense pages had nothing on them but ads for this particular word. I assume they're getting the ads targetted to what they want via mediabot cloaking.
7. Phoned Google, talked to a helpful girl named Kate, who agreed that the clicks looked odd. She offered to pass the offending sites onto the AdSense team, and sent me a form to fill out for information.
Time to resolution: Approximately 15 minutes
The moral of this story is that you need to be measuring everything having to do with your AdWords account, especially considering Google does not break out content clicks within your account. It is quite easy to detect simple clickfraud on a large advertising campaign by tracking and observation. You don't need expensive products.
Maybe:
1. As Patrick says -- shut down the high bidders and corner the market. The ecommerce dollars are all yours as soon as they are booked/billed.
2. Test the sensitivity of the Adsense fraud detection. If the botnet processed those clicks through randomized Google search queries would that referral traffic offset the traffic-surge fraud detectors? Was the offending site a throw-away, proxied/falsified whois, short TTL .biz, etc etc etc or was it a valuable historical domain name?
3.Shut down someone's adsense revenue. Was the site in question SERP dominant in a competitive arena? Was it a scraper site or redirect hijack site sitting on someone else's pile of gold? Was it a dating affiliate site owned by an abusive DMOZ editor? There are many reasons sufficiently valuable if the emotions are involved.
Best part is if the offender has no idea you were paying such close attention. It will appear that Google detected the problem (even though they did not).
Someone mentioned that using a bot with a large list of IPs was a sign of intelligence. That isn't true at all. There are a number of client side software packages on the (black) market that come with a list of working proxies pre-installed. The user just plugs in a list of urls and a few parameters and they're done.
It requires no intelligence or creative thought whatsoever... A reasonably bright monkey could do it.
In years of dealing with click fraud I have found that the majority of it comes from incomprehensibly stupid people.
The only conclusion I've been able to come to is that they do it so obviously because they can get away with it.
They throttle up traffic, get paid for it, and so send a little more the next week, etc...
Eventually someone notices, but getting caught really isn't a big deal. Even the stupidest ones expect to get caught and spread their efforts over multiple websites, through multiple accounts.
There are a lot of great ways to detect fraud using automated systems (many of which none of the big PPCs are using yet) but all of them rely on the people doing the fraud being a little bit stupid.
All I can say is thank God they are. If a significant % of them ever get smart then the PPC model as it exists today is done.
I take it from the response you're only using Adsense.
Nope I'm an adwords user as well as an adsense user. I dont' spend a fortune, but around £10k a month, so not pocket change either.
d
Out of interest, do you use Google's conversion tracking facility on the adverts that had the fraudulent clicks?
It was always my impression that Google were using that to help them identify fraudulent activity on the content network?
For example, an Adsense partner site delivering 0.001% conversion ratio compared with other sites in the sector delivering an average conversion ratio of 2.5% proberly means the site with 0.001% is up to no good.
Out of interest, do you use Google's conversion tracking facility on the adverts that had the fraudulent clicks?
Nope, conversion system is entirely in house. My biggest fear as an advertiser is Google raising keyword costs per vertical, and by advertisers using their conversion system, Google's now got the data to say "ok, we know how much you make, this is how much we can charge and you'll still advertise".
Because if we don't notice...they make even more money that's why... and so does the shady publisher which in turn encourages other shady publishers to sign up. The problem started when Google and Overture threw their doors open to anyone instead of the more respected sites which they started with.
It's just a matter of time before they get hit with a HUGE class action suit for defrauding their clients of advertising dollars and make no mistake...they are ultimately responsible because they are the ones taking the money. That might wake them up... or it will just die a slow death and we'll all jump on to the next best great wonderful thing that comes along.
I think Overture and Google ignoring this area makes them shady too in my book
Why does Google and Overture leave it up to the advertisers to catch fruadulent clicks from shady partner /publishers?
They don't.
Why does Google and Overture leave it up to the advertisers to catch fruadulent clicks from shady partner /publishers?
They do their best to identify it. The real question is why would you leave it up to them?
Let us know what the final bill comes to, i.e whether you get charged for 61 clicks or 6,000.
Also - didn't this kind of increased activity trigger a 'daily budget' spend cap?
Nope, conversion system is entirely in house. My biggest fear as an advertiser is Google raising keyword costs per vertical, and by advertisers using their conversion system, Google's now got the data to say "ok, we know how much you make, this is how much we can charge and you'll still advertise".
I think this has already happened in specific categories in a couple of comparison shopping engines. I also have stated that anyone that gives Overture their conversion data, is only opening the door to minimum bid increases for specific vertical markets in the future.
ClickAssurance (based in Texas)
Whosclickingwho (East Coast)
Alchemist Media (based in Los Angeles)
They probably will only work with folks spending $25K+/mo, but I'd highly recommend trying out these firms to see if they can detect both recurring IP-based (stupid) fraud and proxy network fraud (harder to detect). In some cases these companies will work on a bounty basis, taking a percentage of the dollars they recover for you from Google & Overture.
In general, BakedJake's suggestion that everyone should be tracking cost and conversions at a keyword level is a good one, as it's the first necessary step towards knowing what's really going on.
Shorebreak
All-in-all, not too hard from investigation to resolution, and well handled by Google.
When you find these scrape sites using your content to generate Adsense revenue, you take the time to click the link on the ad and point the site out to Google.
You consistantly get canned responses in your inbox, but Google never seems to do anything to the offending sites. They just ignore it because it produces revenue for them.
So how do you go about getting Google to take action? You develop a bot and click the crap out of the scraper sites, which inturn gets them flagged for fraud. And that tends to get those sites removed.
Google doesn't care at all about the quality of Adsense sites. If they did, they would require that each domain you put your code on would need to be approved before the ads would display.
But Google is very sensitive to NCC's on the content network because it doesn't have the kind of ROI margins that search does.
Suppose you had a site that was in a competitive space. That site was constantly scraped and used by crap AdSense sites.
When you find these scrape sites using your content to generate Adsense revenue, you take the time to click the link on the ad and point the site out to Google.You consistantly get canned responses in your inbox, but Google never seems to do anything to the offending sites. They just ignore it because it produces revenue for them.
So how do you go about getting Google to take action? You develop a bot and click the crap out of the scraper sites, which inturn gets them flagged for fraud. And that tends to get those sites removed.
Google doesn't care at all about the quality of Adsense sites. If they did, they would require that each domain you put your code on would need to be approved before the ads would display.
But Google is very sensitive to NCC's on the content network because it doesn't have the kind of ROI margins that search does.
You're giving me ideas....
I'm not going to resort to putting a bot out to produce fraudulent clicks, but that idea is tempting as a way to get rid of the scraper sites.
You're right that Google doesn't care about the quality of the sites. Many people at Webmasterworld seem to think scraper sites are against the AdSesne rules, but I complain about them all the time and Google lets them continue to operate.
I would recommend the following really put a stop to click fraud. Since I have implemented the below I am saving myself 1/3 of the normal clicks with improved ROI.
Get the Following if you don't already:
www.gotoast.com - ppc bid tracking, and landscape on bids. Free Trial
www.clickdefense.com - Click Fraud & ROI tracking - Free 30 day trial
* Reporting tools to export Click Fraud detailed reports across any keywords, from any search engine. They keep the behavior of click offenders.
Worth a try.
-Scott
www.ClickDefense.com
You're kidding, right?
From what I hear, their system catches pretty much zero advanced click fraud. I haven't heard a single good thing about them.
Please elaborate.
It's against the TOS to mention specific companies. But since you brought up clickdefense.com, I just wanted to mention that I heard that they catch about 1% of actual fraud, and were terrible when tested.
The plant on my desk can do a better job than 1%.
You must be talking about your Venus Clicktrap.
1) You are a new member as of 1/26/05
2) Your first post is pumping GoToast and ClickDefense
ergo, you are likely affiliated in some way, non?
And I must say Blake, your information is inaccurate and unfounded. Perhaps you should phone the Google Fraud Squad, and get some more revealing information about it.
Sure I'll provide Click Fraud information and then let you guys figure it out, how about that? Not. That is a wast of time. You have to link drop to help users. (of course spamming on just ones own site is another story)
Anyway before you toss your thoughts that it's less then 1% why don't you kick it's tires first. They are more then likely going to set the standard for the industry on CF terms.
If you'd like I'll email you what my report looks like, and its catching up to 30% as click fraud traffic.
Anyone promoting a site on the first post is a red flag but promoting it for first 5 post. hmmmm.... All 5 causing FUD ( Fear, Uncertainty and Doubt) regarding click fraud and prodding us to use xyz.com to check it out. Luckily the fourth post which was a much bigger post on this matter and much more overtly advertising xyz.com, was removed by the mods.
Do you think we cannot see through this?
All of a sudden he's acting like he's an equity partner in ClickDefense, and they are the next big thing, since sliced bread...
There are obvious patterns you can find in your logs, but no auditing platform, is going to track random clicks automated through public/private proxy servers. As valid traffic also flows through these proxy servers, how do you filter for what is fraud clicks?
Most of these services are just riding the "hype" that supposedly up to 50% of all PPC traffic is fraud, which is a load of crap..
GAW/Overture represent 98% of PPC traffic. Track ROAS/CPA at the keyword level, and you will automatically take into account all traffic as it relates to profitablity at the engine/keyword level.. If GAW/Overture don't do a good job on their end, either through distribution channel or fraud control, your bleeding is minimized...
The KW is only $.05 and normal spend is about $1. I noticed that the spend went up to more than $25 for 1 day and all from the content network.
I checked out my site briefly and it appears to be fraudulent clicks, although I haven't looked closely at the detailed logs b/c it's not worth it.
I'm going to wait until early next month to see if G actually refunds me w/out me asking. I highly doubt it, but we'll see. I don't see how much more obvious this click fraud can be than have 25x clicks in 1 day from a content partner.
