How To Defend a Google AdSense Site From Click Bombing

[jbayabas]

I may be a victim of clickbombing. I had an unusually high clicks in one of my sites yesterday.


28,520 page views and 437 clicks

This site usually only get 44 click on average.

I don't know what's going on and I don't know to find out if the clicks were valid.

What should i do on my end. I thought Adsense had a technology that filter invalid clicks.

[ember]

99% of my traffic is from the U.S. The last few days I've had IPs from Russia and the Netherlands and a Washington IP that is generating a lot of pageviews every day. I've blocked them all, but I'm still seeing wild fluctuations.

[jbayabas]

The plugin that I mentioned in this thread is quite useful. It's able to track IPs that click multiple times depending on maximum clicks you've set. Example, if I set a max of 2 clicks per ip per day, it stops showing the ad from the user if he clicks the ad more than 2 times. Based on the data it gathered, the problem is not click bombing but a glitch in Adsense reporting.

Netmeg mentioned google has hired an anti click fraud company, perhaps they are testing the new filter, but sadly, it's doing a horrible job.

[dolcevita]

Maybe i'm too suspicious but i'm not sure anymore what is good and what not.

For the last hours statscounter show 65 clicks while Adsense show 35. Browsing through Statscounter logs i've seen 9-10 clicks from the same IP and it looks that Adsense filter works good. I will be very happy if it starts to works as it suppose to be.

But what i found strange is very high CPC and it is trend in the last 2 weeks.

It is 2x higher that it was all time. Maybe it is not topic to discuss about higher CPC but is someone else noticed 2x higher CPC in the last 2 weeks or it is only me?

[superclown2]

I'm no programmer but even with my limited ability I reckon that by using a mixture of VMware and a service provider offering dynamic IP addresses I could put up a scalable system very quickly and cheaply that could send clicks to ads with different referrer data every time. Maybe Google could still detect this but if so I don't know how. It was interesting that a company with billions of dollars to spare and a business model completely dependent on ad clicks should feel it necessary to buy another company devoted to click fraud elimination - what's wrong with the thousands of graduates that Google already employs? It smells of desperation to me.

I gave up on adwords years ago when I found that the major part of my time was being spent analysing my stats and chasing Google (and Yahoo) to credit me for the 40% or so of clicks which were clearly fraudulent. No doubt their detection methods have improved since then but a lot more people have become disillusioned by Google and some of them are brilliant programmers. The surprise to me is that we haven't had many more threads like this in the past.

Would Google admit that they are under constant attack by determined fraudsters or malicious people with a grudge? No way, it could destroy their business so I'd be amazed if any official announcement was made.

[netmeg]

Can you post ip addresses so I can check them in my logs?

No. Pretty sure that would be against the TOS. That's why I didn't mention the host. You need to find your own IP numbers.

Google *just* bought the anti click fraud company; I'd be really surprised if anything other than paperwork has happened so far. Maybe, but I doubt it.

Nope, haven't heard of anyone being banned. And even once the clicks get removed from account (either daily or at the end of the month) I'm still up over last year. But I object to the reporting - while ostensibly we're supposed to have more "up to the minute" reporting, it actually nets out to not knowing how my sites are doing until after the end of the month. Plus because of the batsh*t crazy way Google does things, the end of the month take backs are "adjustments" so for example if I have an extra $900 in my account from last month, even though it never makes it to my bank account, it's still stays in my reported earnings. Nuts. And if these clicks are real, I still wonder about what's happening on the advertiser side.

jbayabas - just to test I put that plugin on one of my affected sites for two days, after I had my developer vet it. Every IP it found - when I went to Analytics and StatCounter, it was very obviously a normal human user. (Two of them even filled out forms) What's more, those clicks did not get removed from my AdSense account, which I would expect if they were invalid. So there's that. YMMV.

[ken_b]

I heard back from Google. They said I'm fine, not to worry about it.

Also that removing the ads "may not be the best decision" and I could consider putting them back on the pages. Then he added that he could see that this channel has been getting a lot of invalid traffic.

So I did put ads back on these pages, but not the same adblock. Instaed of a 120x600 adblock I used a 160x600 on some pages and a 336x280 on the rest. These new ads went into NEW channels, one channel for each size. One channel is set to image only and the other is set to image and text.

Invalid clicks returned immediately. You can see the clicks/earnings come and go by refreshing the stats page every few minutes.

I don't have the time or interest in fussing around with this, so I'll probably just take the ads off.

I don't know if I'm having this issue on other pages/adblocks. But these problem adblocks appear on about 48 pages 1 level down from my home page and 1 level up from the my main 1,500(+/-) pages.

I only noticed this the other day because the earnings number for this channel was way too high when I checked in the morning. (I normally only check stats in twice a day, first thing in the morning and last thing at night.)

.

[dvduval]

One time I noticed a huge jump in both traffic and profits on adsense, and I found some very strange referrals. I reported it to google. I got no response but found my account was reduced to less than I normally make. I went from 1000% increase to 50% decrease for the month. I guess at least I didn't lose the account, but pretty much you are at their mercy, and there is little or no communication possible.

[jbayabas]

Clickbombing is still happening here for me. Can't believe this is still not fixed.

[netmeg]

So after all my poking around, I did a pretty aggressive banning of a couple of ISPs (the worst of which was Amazon AWS) and while the invalid activity hasn't completely stopped, it has definitely slowed down.

Look for visits with very low durations - like 0-5 seconds.

Another marker I noticed - all the suspected bots, no matter what the ISP, reported as Firefox 18, 1024x768 resolution, and as Direct traffic (no referrer). I'm not saying that they can't be human, but you might want to take a look at the behavior of direct visitors reporting with those metrics.

[jbayabas]

When I look at the Adsense stats by country, I see the UK with a big surge. Last time it was either the USA or Germany. I don't see any clickbombing in statcounter, analytics, and the plugin. Really frustrating.

[webcentric]

Well, as a result of this and other conversations, netmeg has been able to define a process for identifying where some of this activity is coming from. Identification of problem IP addresses, IP Blocks and or certain ISP's is a good first step and if your read back in this thread and the Adsense Earnings and Observations thread for March, the clues are there for how to go about putting a finger on some of these culprits. Of course, knowing where the problem is coming from is not a solution in itself. Next you need a way to block the offender if you can. If you have a firewall, great but many on shared hosting don't have access to that level of blocking. Now I'm no Linux guru but I understand that you can block by IP using .htaccess and while this isn't keeping a bot from hitting your server, it can keep them off your pages which may be enough to discourage them from coming at all eventually.

Of course, there's always the fact that the bot can head off and find a vulnerable computer or proxy to come at you from so no solution is going to be perfect but if you can even slow them down by first identifying them and then putting some sort of blocking in place, you're helping to combat the problem.

This conversation has shown that it's possible find out where at least some of this trouble is coming from but it takes some careful detective work. One of the first steps in that process is to capture as much info as you can while it's available because as Google pulls back the revenue, it takes back the related stats with it. So you need to get to that information during that window of opportunity. From there you can use other tracking tools to get to the offending IP address.

As netmeg noted, you're looking for behaviors generally but also signatures.

@netmeg -- just wondering if you think that using duration, user agent info, etc. as you mentioned may be a reliable shortcut to finding these bots or do you recommend taking the longer, more methodical route as you described above? It would be great if there was a simple set of indicators to look for, some sort of reliable signature that could be used to narrow the search. Of course different bots may be using different signatures and a bot can also change its signature so that isn't going to work all on it's own I think. Sounds like duration is your main point of emphasis where a signature is concerned.

[jbayabas]

Netmeg, how do you ban ISP. Do you do it by htaccess?

[netmeg]

Yep. I use an online tool to find the IP blocks belonging to the ISP (there are bunches of tools out there) and then deny the block in .htaccess.

In the case of something like Amazon, I am temporarily taking out two entire blocks like this:

deny from XXX.XXX.

Which denies a BUNCH of IP numbers, so it's aggressive, possibly overly so, but I wanted to see what kind of impact it would have. It's only been a day; I'll leave it there at least a week before I decide to leave it there for good.

@netmeg -- just wondering if you think that using duration, user agent info, etc. as you mentioned may be a reliable shortcut to finding these bots or do you recommend taking the longer, more methodical route as you described above?

Depends on the tools and time you have to devote to it. And duration can be tricky - some of the bots I notice that come back over and over report ridiculous duration times - like two hours - so you have to be careful.

The first important metric seems to be Direct traffic. So far, I've never seen an obvious bot that reports as coming from another website or from a search engine. That'll probably change some day, but so far, my bots don't seem to be spoofing that. So that's the first way to narrow it down. Start by looking at Direct traffic.

Then you want to look for anomalies. Like for example, all these bots report a browser of Firefox 18.0. The current version of Firefox is 27.0.1. They all report at 1024x768 resolution - what real screen even uses that resolution nowadays? Not laptops, and not 19" and larger flat screens - but every single one of these log entries says 1024x768. To me, that smells like some kind of black hat tool for scraping/clicking, so that might bear looking into.

I have other bots (that look like infected PCs) that all report as IE, but all different versions and resolutions. That particular anomaly is that on a site with 3400 pages, these bots only hit four pages, and never hit more than one page per visit. So for these, I was able to set up a segment of direct traffic, that visits page 1 OR page 2 OR page 3 OR page 4, and has IE has a browser, and the visit is less than 5 seconds.

So for me, I look for the signature first, and then I look for the behaviors within that signature.

[dethfire]

I usually see clickbombing early morning. I'll shoot up in earnings real quick and then for the next hour it will decrease.

[ember]

I also found an Amazon IP using FF. No referring link. Resolution unknown and pages visited unknown. And another with no referring link and unknown exit and entry pages. The unknown part is what looks weird.

[dolcevita]

netmeg...maybe you can post amazon ip ranges that you have banned.

[netmeg]

I don't think that falls under the charter or TOS of this forum or WebmasterWorld. Besides, just because they're hitting me doesn't mean the same ones are hitting you. (As it is, they're only hitting my three oldest sites, not any of my newer ones.) You need to look at your own logfiles to see what's there.

[jbayabas]

Netmeg may have found the culprit with Amazon ISP. Pls give us an update in a few days to confirm.

[sirius]

It first I went crazy, blocking all countries except USA which my site is for. All amazon, dynamic ips, etc.

It has done nothing to stop the attack.

I have setup a monitoring system to monitor every ip address.

When the click bombings start, there is no influx in clicks or new ips coming to the site. Most of my traffic is logged in members.

All clicks are originating from USA.

At first I noticed a few user agents that were odd, but it could be something or somebody totally unrelated.

if(preg_match('/Mozilla\/5.0 \(X11; Ubuntu; Linux x86_64; rv:(.*)\) Gecko\/20100101 Firefox/is', $test)){

Nothing has seemed to have any effect. When the attacking starts it starts and there's nothing that can be done about it as far as my end I believe.

[webcentric]

@netmeg, while searching I came across this WebmasterWorld thread and am wondering if you see it as relevant to this discussion where AWS is concerned?

[webmasterworld.com ]

[sirius]

Here is the amazon cloud network ip list, if you want to try. [docs.aws.amazon.com...]

Here is one on the list spidering my site

54.72.0.0/13
54.80.0.0/12

[jbayabas]

Sirius, did the clickbombing stop when you removed the affected ad unit(s)?

[jbayabas]

I wonder if the Amazon ip is a robot that display their ads? I sometimes see amazon affiliate ads on my Adsense ads. They may not be the culprit after all?

[sirius]

jbaya, I don't think so. It's hard to tell. Even after taking Adsense completely off, clicks and impressions come in for hours. Stuff is going up and down left and right. Today it seems to be stuck way up at $250 even with my site taken from the authorized list.

[sirius]

The Amazon IPS are Amazon Cloud servers. [aws.amazon.com...]

Anybody could use them..

[jbayabas]

Sirius, I wouldn't advice you to remove Adsense from your site else youll earn $0 money. The problem seems global and it's beyond our control.

[sirius]

JBaya, it's not. When the bombings go crazy I just take my site form the Authorized list. I'd rather loose earnings until they can fix it than piss off all the advertisers who pay us

[netmeg]

First of all, there isn't ONE problem. I'm still getting inflated clicks, I'm just getting fewer of them at the moment.

So blocking Amazon probably won't solve all your clickbombing problems. Personally, I still think there may well be a reporting issue as well. Also, interestingly, the IPs I blocked from Amazon are not on the list mentioned above, nor included in Bill's post in the Spider forum. I'm not sure if AWS is the same as the Cloud servers, but they're definitely Amazon numbers. Maybe they're new.

So ok, since nobody seems to be objecting (and people are posting IP numbers in the other forums,) this is the range I block. Again, YOUR mileage may vary.

deny from 54.186.
deny from 54.213.
deny from 54.218.

90% of my Amazon visits for the past month (tens of thousands) came from those ranges. I couldn't see a good reason NOT to block them, so I did. (I did catch one very blatant set of clicks coming from those IPs, and got a screenshot of it out of my Google Analytics, matching it up to my StatCounter report)

They're only coming to my three *oldest* sites, they're not coming to any of the newer sites. So either those sites aren't on "the list" (even though there are links between them) or they started this before I had the newer sites or they're attracted to the highest traffic sites - I dunno.

But I'm going to leave it there for a week and see if the noticeable drop in invalid activity maintains.

[ember]

Wild fluctuations this morning, the worst yet. I found an Amazon bot with IP 54.85.18.197 and blocked it. Knock on wood, since then things have settled down and almost look normal. Time will tell.

[sirius]

Searching Google found this [imageshack.com...]