How To Defend a Google AdSense Site From Click Bombing

[jbayabas]

I may be a victim of clickbombing. I had an unusually high clicks in one of my sites yesterday.


28,520 page views and 437 clicks

This site usually only get 44 click on average.

I don't know what's going on and I don't know to find out if the clicks were valid.

What should i do on my end. I thought Adsense had a technology that filter invalid clicks.

[ember]

Yep, it's happening again.

[wa desert rat]

Yep, it's happening again.

Check for xlhost.com visits.

WDR

[jbayabas]

I don't see any amazon ips this time. I see lots from Verizon though but i'm not sure if they are legit or not.

[netmeg]

Yea I blocked xlhost a couple days ago too.

Like I said before, there's no *one* solution. You have to look at your Direct traffic and see if it makes sense.

[wa desert rat]

Verizon are probably mobile devices (smartphones, tablets). So far I have had only one click from a mobile device pulled back so I don't think they are involved (yet).

Look for direct entry, one page (100% bounce... check your "network" entries in Google Analytics) and it appears that they are listing "firefox 18" as their browsing agent. All, so far, Windows operating systems in my case - or that I think I've found.

At this time I see only two domains with 100% bounce rates as listed in the "network" part of Google Analytics.

Right now I'm stable... one click shaved off but that could be an accidental click.

WDR

[jbayabas]

found one.. IP: 88.208.201.208

ISP:Fasthosts Internet Limited - Uk's Largest Web Host

[wa desert rat]

xlhost.com has a larger IP range than I posted above. Here is the new CIDR: 173.45.64.0/18

WDR

[wa desert rat]

found one.. IP: 88.208.201.208

ISP:Fasthosts Internet Limited - Uk's Largest Web Host

CIDR notation for their entire assigned IP range is: 88.208.192.0/18

That particular IP reverses DNS to: server88-208-201-208.live-servers.net so search your logs for "live-servers.net". You can use a subnet calculator (google for one) to make that ban smaller if it's needed. There might be some good systems on there. It's a huge hosting site.

WDR

[Lame_Wolf]

Lame Wolf, I blocked wowrack, IP 208.115.111.74, but they keep getting in. Did you block a range?

I just added the ones I found...

208.115.111.66
208.115.111.67
208.115.111.71
208.115.111.74
208.115.113.82
208.115.113.83
208.115.113.87
208.115.113.90
208.115.113.92
208.115.113.93
208.115.113.94
208.115.124.133

[wa desert rat]

wowrack.com
CIDR: 208.115.96.0/19

Not sure you want to ban all of them though. But if I found the ones Lame Wolf found I would ban that entire /24 subnet.

If you use Linux then you can search this stuff out with a simple: "whois 208.115.111.66" on the command line. Then a "nslookup 208.115.111.66" gives you their reverse lookup (because it doesn't always correspond to the forward lookup and your server usually logs the reverse version so you can search for more).

WDR

[webcentric]

I keep coming back to the humans vs machine point where whether to ban a hosting provider or not is concerned.

Adsense publishers want human visitors. Websites/web applications/bots/etc. do not generate legitimate clicks, people do. Applications scrape pages, steal content, spam our sites, and mess up our reporting, for starters.

Applications come at us from IP's provided by their hosting environments or proxies (at least a great deal of the time), people come from IP's provided by ISP's for the most part. In general, a hosting company is not sending human traffic (except via referrals from sites they host). Blocking the hosting company's IP addresses is not the same as blocking the human visitors their sites might refer to us (at least it looks that way to me). I'm still looking for a good reason to not just ban traffic from every hosting company on planet earth. They're hosting applications and websites, not providing people with access to the Internet (for the most part). Of course, such blocking would be a pretty daunting task and I'm just making the statement as a way of putting traffic sources into a more proper perspective. I'm also not pretending to have a complete grasp on the ramifications of say, blocking Amazon AWS, but I am asking the question, what impact on your human traffic is there going to be as a result of banning all the applications, websites and bots operating in a specific hosting environment?

[not2easy]

There is an entire forum here devoted to UserAgent and spider identification. Once you know the server you're trying to block you can use the WWF Search and save time finding what CIDR to block, and the IP range it includes.

The traffic from another site will show their own ISP IP, not the IP of the server they came from. I block servers from all around the world, Amazon in particular. Yes, it is an ongoing task, but not monumental once you get a handle on it.

[themoabird]

found one.. IP: 88.208.201.208

ISP:Fasthosts Internet Limited - Uk's Largest Web Host

I blocked that IP a couple of months ago. Instantly stopped some very bizarre, very high paying clicks (which were always removed) - i.e., $60.00 per click.

[jbayabas]

What? You mean this has been going on for 2 months and google still hasn't found a fix?! This is ridiculous!

[webcentric]

Regarding user agent and spider identification, here's a thread from the above-mentioned forum that's as good a place as any to start.

[webmasterworld.com ]

What? You mean this has been going on for 2 months and google still hasn't found a fix?! This is ridiculous!

Your contention that this is something for Google to fix is completely off-base IMHO. They ARE taking steps to protect their advertisers. Should they be more up front about what's happening? Perhaps.

But "THIS" has been going on since the inception of the World Wide Web. Part of the job of being a webmaster is to learn how to defend your website. And that job will never be over. Believe me, I got a good taste of that lesson myself recently. What doesn't kill us makes us stronger, no?

[themoabird]

What? You mean this has been going on for 2 months and google still hasn't found a fix?! This is ridiculous!

Longer than that. The strange clicks started November last year. At first I thought it was just a reporting glitch - because it always showed up as a single click, and surely a single click could never be worth $60.00 - but then in the new year it occurred to me that maybe it was a bot clicking multiple times, with Google only reporting it as one click, so I started searching through my logs. Hard to track down, because the clicks never stuck around long enough to end up in Analytics, but I eventually worked it out, blocked the IP address, and the problem went away. (Though I too have been suffering from the Amazon bot issue - fixed for now.)

[ember]

So in analytics - > networks -> I am looking for visits that view one page, have a 100% bounce rate and just stay for a second?

[jbayabas]

So you mean to say we should waste 90% of our time watching our stats to defend ourselves from these bots while we spend the remaining 10% of our time creating good content. That's my life now for the last few weeks. Ahhhh!

[netmeg]

So you mean to say we should waste 90% of our time watching our stats to defend ourselves from these bots while we spend the remaining 10% of our time creating good content. That's my life now for the last few weeks. Ahhhh!

That's part of the job, same as watching your traffic, paying attention to copyright issues, scrapers, keeping your site secure and hack-free, and site performance. Same as if you had a brick and mortar business, in addition to selling you have to pay attention to accounting, keeping your store clean and tidy, people trying to shoplift, etc.

Some months it goes worse than others, but yea - that's what being a publisher and running a business *is*.

[wa desert rat]

It's not that onerous. Several of us have spent a lot of time over the past two weeks trying to track down the major issues and keep everyone else on here informed of our progress. You're getting details about the bots we find and details about how to deal with them.

It's up to you to search out, install and learn to use other tools to keep your site earning money for you. This would include different analysis applications, methodologies to use to identify the source of problem visitors, emailing abuse listings, etc.

You might also benefit from learning to use Linux as part of your toolkit. Several of us have Linux as our regular desktops but others simply dual boot or run it in a virtual window. I think that the simplest method is to put Linux on a machine that is no longer in regular use and maybe not quite robust enough to run Win7 or Win8 but plenty good for Linux and then use a KVM switch to work between the two.

And yes, I know it's a lot of work for a site that might not reach the minimum payment every month. But, as netmeg says, it's part of the job.

WDR

[netmeg]

(of course, some of us prefer FreeBSD to Linux)

[wa desert rat]

(of course, some of us prefer FreeBSD to Linux)

Yes, for those of us who haven't been able to move on. :P

WDR

[Chris13]

There have been lots of reasons to block Amazon AWS, Amazon EC2 and xlhost (formerly ee.net) for some time.

[webmasterworld.com ]

[webmasterworld.com ]

If you do a google search for "scraper AWS" you'll find lots of hatting material on how to set up free proxies and how to set up a content scraper using AWS's functionality.

A scraper can easily make a list of pages with adsense ads. If you search your older logs, you will find all of the ip addresses you've been having trouble with the last couple of weeks have probably been visiting your site for months.

Check out the search engine/spider forum.

[webmasterworld.com ]

[lzr0]

A scraper can easily make a list of pages with adsense ads.

This is an easier part. You just need to authorize only your own sites in AdSense control panel.

[lzr0]

@ember

So in analytics - > networks -> I am looking for visits that view one page, have a 100% bounce rate and just stay for a second?

Not neccesarily. In my case the spammer's robots I found had different patterns. One was visiting my site periodically every hour or so during entire day and statcounter reported hourly-long visits. The other one was visiting at exactly the same time every day. In both cases there were no referral urls and no exit urls.
So, I am persoanlly looking for repeated visitors who keep coming either at exactly the same time every day or keep coming during entire 24-hr period.

[netmeg]

The first way I'd narrow it down is to look at Direct traffic. No matter what I'm looking at, all the bot stuff seems to be Direct (or no referrer).

[Chris13]

This is an easier part. You just need to authorize only your own sites in AdSense control panel.

Not exactly what I was referring to. Of course you should only authorize your own site(s), plus google.

I was referring to the bots building a large database of web pages that have adsense ads on them. Once you have enough of them you can unleash the bot and have it go the pages one by one and do your deeds with the ads.

That's more efficient for a bot that randomly roaming the internet looking for ads to click on, which would generally mean crawling sites and checking links and pages. Too slow, too inefficient to have a broad scale affect, and more easily detected as here comes a bot crawling your entire site.

These guys began with a large list of pages containing ad sense ads, as witnessed by the breadth of the attack and the fact that they went straight to the page.

I was a bit surprised to learn that so many webmasters hadn't already blocked troublesome server farms. This is just another lesson to learn. I blocked them because they were scraping my site and had scraped 10s of thousands of pages. I didn't know what they were using the data for. I only knew I didn't want them hitting the server. I became a lot more aware of combatting scrapers and bots.

[jbayabas]

Question about bots.. When I look at the amazon IPs paths, I don't see any ad clicking so why are we blocking them again? How do we know they are responsible for the Clickbombing when we don't see any clicking in the stats?

[netmeg]

Maybe they aren't on your account. They definitely were on mine, and I confirmed it.

(I don't know what you mean by "look at the amazon IP paths" - if you're talking about looking at them in something like StatCounter, you won't see them because StatCounter does not track AdSense accurately)

Ran into my first issue with blocking all the Amazon AWS - some parts of my Raven Tools no longer work. I'm going to see if they know the specific IP range so I can unblock it, otherwise I'll just have to do without those tools for now.

[eek2121]

Some extremely sketchy stuff going on. Today my CTR was triple (on one of my sites) what it usually is. According to analytics all of the traffic that clicked ads came from google, and they are scattered all over the world and all on different ISPs. I'm going to be keeping a close eye on things tomorrow.