Click Fraud by Bots Could Cost $6.3 Billion in 2015

[engine]

So, here's another look at the attack on online advertising spend. Bots are one of the culprits this time, and it seems there's little that can be done about it right now. It's not just Google that's impacted, of course: Everyone involved in the ad business, including Microsoft, Yahoo, and advertisers big and small. Publishers are at risk, too, as they see their ads earning less and less.

The report indicates that in 2015 bots could impact advertisers by as much as $6.3 billion. Of course, that just an estimate for the year, and takes into account what appears to be a bot, rather than a human. It's challenging to get precise figures, but, with the rise of bots, advertisers may have to accept there's going to be some loss involved in their online advertising.

Not only is there a problem with ad blocking software affecting publishers [webmasterworld.com] and advertisers [webmasterworld.com], but mobile, the biggest growth sector, is particularly challenging to get a good ad performance simply though the technicalities, and fat finger syndrome.

The most startling finding: Only 20 percent of the campaign’s “ad impressions”—ads that appear on a computer or smartphone screen—were even seen by actual people.

“The room basically stopped,” Amram recalls. The team was concerned about their jobs; someone asked, “Can they do that? Is it legal?” But mostly it was disbelief and outrage. “It was like we’d been throwing our money to the mob,” Amram says. “As an advertiser we were paying for eyeballs and thought that we were buying views. But in the digital world, you’re just paying for the ad to be served, and there’s no guarantee who will see it, or whether a human will see it at all.” Click Fraud by Bots Could Cost $6.3 Billion in 2015 [bloomberg.com]

In this report, researchers claimed that Google [webmasterworld.com] has been charging advertisers for YouTube ads, even if it suspects an ad was viewed by a bot.

This really is becoming a major issue for the advertising industry, just as it has been for many years with site statistics being skewed.

[incrediBILL]

Good.

People dismissed bot blocking so let them pay.

The costs are actually much higher in other ways to numerous to go into, but nobody can say they weren't warned.

The open architecture and haphazard construction of the web is responsible for this mess and the only true solution are IPs known to be assigned to humans, and discard all the rest.

Even that has problems as hackers just build botnets in the machines of humans to bypass such controls. The next problem(s) to solve is to harden the OS to only allow authorized software. Mom and pop don't need a fully programmable OS. There will need to be a developers edition that allows anything but the rest of the population need to be risk free, hacker proof. I believe it can be done.

The botnet traffic also needs to be blocked at the ISP once and for all. Shouldn't be that damn hard.

Until the hard decisions are made, and the hard steps taken to fixed it all, let them pay.

[mcneely]

What a difference a day makes, huh?

Here we've gone from not enough clicks (adblockers) to too many clicks (bots) -- Villains on both sides of the fence these days and the ad networks and publishers are somehow powerless to stop it all. Imagine that.

Out of it all, those pesky bot clicks are probably the easiest to control. I wrote a deal way back when addressing this very subject, but it was having more to do with scrapers than clickbots - I'll give the same advice now, that I did way back then -- Block the bots and blacklist the offending IP's --

Place your ads responsibly and watch your 6 with regard to the bots ...

[RhinoFish]

My idea to solve all of this, keeps being dismissed, its too disruptive (and lengthy).

[RhinoFish]

Having unabated perps, doesn't help things. :-)

[netmeg]

Don't get me started on bots.

[Leosghost]

Good.

People dismissed bot blocking so let them pay.

So..you think that the advertisers ( not the ad agencies ) most of whom ( including all the "mom and pops" who advertise on google et al ) have no idea what a bot is..should pay..? and you think that is "good"..? ..They should pay because the search engines and the adsense publishers and the sites that run the ads dismissed "bot blocking"..?

It isn't the advertisers responsibility to block bots clicking on their ads ( which they pay for ), which are on other peoples websites..

It is those who sell the adspace, the ad networks, Google, yahoo MS and the rest who should pay..fines ( not talk about loss of potential earnings )..because they are selling a product ( adspace ) which is not "fit for purpose"..Because they could make it so, but to save themselves money, they don't try hard enough, because it isn't their money that they'd be saving by preventing bot click fraud properly..

Bot click fraud makes them a great deal of money ..even after allowing for refunding some advertisers when the PR calls for it ..

[IanCP]

The report indicates that in 2015 bots could impact advertisers by as much as $6.3 billion. Of course, that just an estimate for the year, and takes into account what appears to be a bot, rather than a human.

If we consider the estimate of $6.3 billion is near enough to factual, then surely there must be ways and means of determining who the recipients are?

Is it being hidden because it is spread across thousand, or hundreds of thousands of accounts?

Over ten years ago, one of our more nuanced Amazon affiliates detected that certain BHO's were intercepting links to Amazon and inserting their own affiliate code. It took quite awhile for many of us to convince Amazon this was happening. So we asked them the very obvious [paraphrased] question.

"Among your top affiliate earners, on close inspection, what doesn't add up?"

A few weeks, months later... "Whoops... You're right. Accounts were closed, legal action is being taken". None of us were compensated though, because it couldn't be identified who lost what.

The system was then tightened up. If, somehow my affiliate link appeared on your site and was legitimately clicked upon, sale completed - I would earn no commission. While I believe there has been some relaxation of the rules, the underlying principle is Amazon can trace back every cent they pay out, as legitimate income earned by their affiliates.

Amazon, being Amazon - watch their affiliates like a hawk. Big earners we believe are routinely scrutinised with a microscope, if only so Amazon can learn what that affiliate is doing so much better than themselves. New affiliates with improbable incomes similarly receive close attention.

While I'm sure AdSense for example have somewhat similar systems in place, I personally doubt they can't track bots by following the money.

We've all had bot attacks, and had clicks clawed back - they are simply malicious attacks because the bot doesn't derive revenue.

As they say in the trade: "Follow the money"

[mcneely]

And speaking of bots --

Advertisers are charged in AdWords for views despite the fact that YouTube has already flagged them as fraudulent.

[marketingland.com...]

Classic Google

Something here tells me that these ad marketers like to talk out of both sides of their face, with Google leading the charge.
We've got stupid Webmasters not taking the necessary precautions with regard to protecting themselves from the clickbots -- Publishers smearing ads all over the wall and in your face thinking that it's the right way to do things (because someone somewhere said so), and an ad industry crying like a bunch of little girls about the money they aren't getting all the while becoming ungawdly rich by taking bytes out of both ends of the pie.

... and you guys really should try to pardon me here for sounding so skeptical - But C'Mon .. I've had guys come to me and complain about Google shaving the percentage of clicks before a big pay-out and asking me to fix it -- Writing in a decent htaccess in order to block a lot of their clickbot traffic only caused them to get their noses all out of joint, because suddenly their traffic tanked.
They couldn't somehow make the connection that they didn't have any real traffic in the first place, and, that writing out the clickbots would help to improve their Adsense earnings over all.

Now, on the other end of the scale, we have folks like Google charging the Adwords guys for fraudulent clicks -- and you know, just like I do, that if they're doing it over on Youtube, chances are pretty good that their doing it on the publishers sites as well.

And who comes up with these dollar amounts any way? -- Billions? Really? ... What with all that's going on lately in the ad industry, I'll have to say that the only thing that's skewed here are the dollar amounts ... Ad industry trying to make things look worse than they really are because the model they're using is old and tired and abused and they really don't feel like coming up with something new.

[Leosghost]

Sure there'll be some exaggeration..after all it is ad men speaking and writing this stuff..but

I'll have to say that the only thing that's skewed here are the dollar amounts

As someone who worked in some very large ad agencies before the days of the web..
much of it does indeed run upon
""marching powder" from the bodies of nubile young things"
which was never cheap..and has not got any cheaper..( or so I have have heard ;)

[IanKelley]

with the rise of bots

This is an issue that deserves attention, but it certainly isn't new. Click fraud was, if anything, more prevalent in the early days of PPC when detection methods were less evolved. Back then there were many more ways for black hatters to generate undetectable fake traffic. And, in terms of percentage of total volume, there was A LOT more of it. People just had no idea.

Since then we've come up with ways to detect almost all forms of bad traffic. Off screen popups, invisible iframes, proxies, bots (real bots, not what this article is calling bots, which are more commonly called zombies) none of is undetectable by the big providers these days.

To my knowledge there's really only one form of fraudulent traffic that really works and that's malware infected computers, i.e. zombies. Even zombies are detectable under scrutiny after the fact because they don't behave quite like humans and they don't convert.

That said, there seems to be some enforcement laziness lately. Google used to be really good about finding and disabling sources of bad traffic relatively quickly. Yahoo (followed by Bing) was a lot slower but they generally got around to it eventually.

All of the above have the tools to make sending them bad traffic economically unfeasable. I don't think they're putting enough effort into using them though. Certainly there is a lot of pure crap making it through from Adwords 3rd party partner sites right now. I hope articles like the above will help to wake them up.

The botnet traffic also needs to be blocked at the ISP once and for all. Shouldn't be that damn hard.

The great firewall of the west? What you would be blocking these days would be the IPs of the ISPs own customers whose computers are infected.

Serves them right you say? Maybe, but it's just another arms race. ISPs would become responsible for detecting malware and blocking it, meaning they would need to deep scan all packets. Putting aside the privacy issues, there's still the cost, which would be significant. We know ISPs aren't going to give up their absurdly high profit margins unless someone forces them, so those costs would get passed on to consumers.

But the problem still isn't solved even then because malware is always one step ahead of detection. It would certainly decrease it's prevalence by at least blocking the outdated malware, but it could never stop it.

Bot click fraud makes them a great deal of money

This always gets brought up in click fraud conversations. And it's very true at the low end of the spectrum (advertise.com for example). They make all their money in crap traffic.

But at the top, Google and Bing, they make their money in conversions. It doesn't matter that they charge for clicks, the business model is based on conversions. If the traffic doesn't convert, or the conversions cost too much because the traffic is being diluted by fraud, that's the end of the party.

They know this and have always worked hard to keep quality up. But like I said, I think there's been a dip lately. Maybe they're too big for their own good at this point. Perhaps some short sighted people in the organization really are thinking that the dollar signs are worth looking the other way for... Those would be very dumb people but it's possible. If so, as soon as someone higher up who understands the industry finds out what's going on, those people are going to be out of a job. If a top tier PPC provider can't deliver quality the money goes elsewhere.

[Leosghost]

In many markets G are the only game, ( U.S.A 60% or more search market share, most other countries outside of Russia and China 90% or more search market share ) the others do not have "the eyeballs" to sell..so for many advertisers there is "nowhere else to go"..

The big search engines are now beholden to quarterly revenue and profit reports..which must show not just growth, but substantial quarterly growth..yes they have dollar signs in their eyes..

Someone higher up ? who..the legislators all have "search histories"..

The regulators are in a revolving door system ( between the C-suites of the search engines and the ad agencies and the offices of the regulators ) with those who they are supposed to be regulating..

The smaller 3rd party networks are as you say scamming away as fast as they can get the money into the bank, preferably offshore, or into all paid for real estate, at home or elsewhere..

The big ones can buy ( or promise non exec board positions to politicians and regulators ) or subtly hint "at individuals search engine histories being leaked" their way out of any investigation..

[IanKelley]

No doubt you're right that regulators are more concerned with their own interests. But no advertiser is going to keep spending indefinitely on a campaign that doesn't convert. If you cut out the advertisers in Adsense who aren't in it purely for branding, there wouldn't be much left.

In this case the regulator is simple economics. There is always going to be some fraud but they can't afford to let it get past a certain threshold.

Well, unless revenue from the partner network is such a small percentage of their total that they don't care about it. I for one would never use it after some recent tests.

[nomis5]

Those would be very dumb people but it's possible. If so, as soon as someone higher up who understands the industry finds out what's going on, those people are going to be out of a job. If a top tier PPC provider can't deliver quality the money goes elsewhere.

Never underestimate how short-term minded those at the very top of huge organisations can be. Who in their right mind would ever believe what has happened at Volkswagen, the largest car producer in Europe.

Even in their maddest moments those guys must have understood they would be caught out. That didn't stop them cheating though. There is simply nothing that can overcome the greed for short term, easy money.

[tangor]

Am I missing something?

Webmasters control entry to their sites (at least the vigilant do!).

I do that even on sites that do not rely on advertising (ie. adsense)

Automated traffic that is not a human has no benefit to me, thus NUKED at the front door.

On those sites were I do run third party adverts, this has stood me in good stead.

If you don't watch the front door, don't get bent when something is "stolen" or damaged (your ad income)

There's more to running a website, kiddies than just pasting ads.

[incrediBILL]

If you think your site is bot free just because you have some bot blocker you're in denial.

People think they know what a 'bot' looks like and you can only tell the ones that want to get caught. You need to learn more, the bots and botnets are nearly undetectable.

Most browsers these days can easily operate as a headless browser so a bot can run javascript, make key strokes, mouse moves, completely fake everything if they want. Basically any computer can be infected and made to dance like a puppet from the master, clicking whatever he wants.

Nobody paid attention, nobody heeded the warnings, and they're paying for it.

Lastly, not all click fraud is automated. There's places where you can hire teams of people using proxy IPs to click on things all day long.

The whole click fraud operation is so sophisticated that I can't stop if from happening without blocking real people as well. I've studied some very obvious bot traffic that had no way to effectively stop short of using a country specific firewall, such as ALLOW USA, DENY ALL which still doesn't block USA bot traffic but the bulk, on the site I worked on with this issue, was USA local only so international traffic was meaningless which is how we knew it was crap.

Other sites aren't so lucky.

[Leosghost]

the advertisers are paying for it Bill, not the website owners..."and that ain't right"..which is why the search engines and the website owners are not really interested in doing anything about it all..

re the countries..yeah..I had two companies that I deal with on totally different matters, offer to put me in touch with their friends or family who run human click teams..they run proxy servers in the U.S.A..they are from a lot further east..it is a quite big business in their country and in other countries, still further east from them..

[blend27]

he-he.

So this is where advertisers should say thanks to AdBlockers for having the software installed. No Ads = No Fraud.

......Finally, one of the mice suggests they go get "Speedy Gonzales." Most of the others haven't heard of him but those who have all describe him as "the fastest mouse in Mexico." One guy says, "I can contact Speedy because he's a friend of my sister." A wise-guy mouse remarks, "He's a friend of EVERYBODY's sister!

just kidding.....

[Leosghost]

Surely the best mouse for click fraud wouldn't be speedy...too easy to spot..even under the sombrero ..apparently randomly dispersed mice however ...

[IanKelley]

No Ads = No Fraud

Joke aside, a user with click malware on their computer and an ad blocker in their browser wouldn't see ads themselves but the malware would have no problem generating fake clicks. So it sucks for advertisers twice.

[shri]

Guys,

A very naive newbie question: Who is making money of these bots?

1) If the ads are being clicked on my website (say adsense etc) then I would be getting a piece of the 6.3B$ pie. I'm not, on high traffic days, I regularly see large clawbacks (10-30%) on the hourly runs. My revenue has dropped significantly on AdSense etc (lower CTRs, lower CPC).

2) If the ads are being clicked on Google SERPS, Google benefits.

So, how do the bot owners benefit? If they're running websites on which they send their click bots, I'd imagine some amount of due diligence on the part of the ad networks / advertisers would isolate these sites quickly.

Who is making the $6.3 billion a year that these advertisers are paying for?

Even at a hugely inflated cost of US$ 1 / click - the amount stated would require 6.3 billion fradulent clicks - not a trivial amount by any means!

I'd like to see how this would go on and not get caught by the networks.

[IanKelley]

Most often the fraud clicks are being sent through a higher tier provider where the fraud control isn't as good.

Some of these get ads from Bing or Google (think Google's "partner network").

You're right that they get caught, eventually. Their goal is to make as much money as they can before that happens. Once they get caught they either switch providers or start over with another account/site.

Generally the providers that syndicate Google ads have to be more diligent because G is better at catching fraud before the first payment. The ones that don't use G or Bing are often pretty lax, allowing a lot of questionable traffic at very low CPC rates.

As far as the hige estimate of how much fraud clicks will cost in the future... it's pure guesswork, they don't have access to any of the actual numbers.

[keyplyr]

the advertisers are paying for it,,, not the website owners...

Hey - I'm a website owner and I paid a whopping 4 bucks last month due to click fraud :)

[Leosghost]

who did you write the check to ?

[keyplyr]

Mr G

[Leosghost]

you paid him..really..you wrote him a check ?
or he paid you what you expected ( potential earnings ) minus $4.oo due to clicks that he didn't like the look of..?
I never had to pay Mr G anything..cos I'm not an advertiser..
He always pays me..cos I'm an adsense publisher..

I never have to pay anyone as a result of click fraud..certainly not Mr G..no adsense publisher does..ever..

[mcneely]

I never had to pay Mr G anything..cos I'm not an advertiser.. He always pays me..cos I'm an adsense publisher..

Sure ... right after Google takes away 8-12% of your earnings due to click fraud -- Regardless of what you think, you're still paying for click fraud. Publishers can get just as soaked as everyone else does.

[Leosghost]

Sorry , but no..
Click fraud happens.. to an advertiser..money goes from their bank account..
But..no money goes out of the publisher's bank account..just less money arrives in it..
As any accountant ( and also any tax inspector ) will tell you, that means that advertisers pay for click fraud..publishers do not..

you are not only counting your chickens before they are hatched..you are saying that the ones that don't hatch are being stolen from you..

Adsense is paid monthly , not daily..only the final payment that arrives in your bank is your true earnings..

That some adsense publishers think like you and some others do, is one of the major reasons why I would never ( if I was advertising ) use the "content network"..

Click fraud can ( and does in some cases )clean out a an advertisers bank account..( they can and some do, go bankrupt because of it ) all that will happen to the publisher's bank account.. is it will not grow as fast as the publisher thought it would..

As publishers we get paid per validated click..not per click..if ( as I do ) you only look at your account rarely , in my case maybe only once per month, just prior to payment in progress..you'd not consider the process the way many of you apparently do..

Many publishers would also be waaaay less stressed..and never be tempted to spend money that they haven't yet had arrive in the bank..

[IanCP]

Click fraud happens.. to an advertiser..money goes from their bank account..
But..no money goes out of the publisher's bank account..just less money arrives in it.

.
My understanding has always been that any clicks clawed back on my publisher account are credited back to AdWords users accounts.

In accountancy terms [I was one once long ago] - I get debited, they get credited - nobody wins, nobody loses.

As for the bots? A couple of us have asked the question without satisfactory answer, who are the beneficiaries of ripping off AdWords advertisers?

Having a bot click on my site, even if undetected by Google seems pretty pointless to me, unless the intent is purely malicious - "we do it because we can cause havoc" - mentality.

[IanKelley]

The profit comes from clicks routed through their own sites. The bad clicks you see as a publisher are incidental. All manner of bots roaming the net.