homepage Welcome to WebmasterWorld Guest from 54.226.235.222
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Google / Google AdSense
Forum Library, Charter, Moderators: incrediBILL & jatar k & martinibuster

Google AdSense Forum

This 46 message thread spans 2 pages: 46 ( [1] 2 > >     
Bots Artificially Inflate AdSense Impressions
Ask, Snap, Alexa and Others Trigger AdSense Ads
incrediBILL




msg:3396087
 6:28 pm on Jul 16, 2007 (gmt 0)

Conventional wisdom has been that most web crawlers don't run javascript so therefore the spiders won't artificially inflate your AdSense ads.

That's no longer true.

The new trend is to augment the SERPs with screen shots in places like Ask, Snap and others, and the screen shots are made with javascript enabled so that javascript page elements (menus, etc.) are visible. That means your AdSense ads are also being included in those screen shots and possibly skewing your statistics.

Both Ask and Snap make screen shots of various pages in your site so it's not just a single home page image we're talking about. Snap has attempted to make upwards of 40K screen shorts on my site alone before I stopped counting. Ask seems to be a little more conservative in the quantity of their screen shots, but that could be growing pains as it remains to be seen if they'll screen shot entire sites and/or all sites they index.

What's unknown at this time is whether or not Google discounts the impressions being made by these screen shot bots at Ask or Snap, or whether those screen shots contribute to your lower CTRs and whatever penalties (smart pricing?) that may bring.

Imagine the impact it would have if a low traffic site with maybe 1,000 pages suddenly had 1,000 screen shots made all at once.

You would see a traffic surge with very low CTR, it would look abysmal.

Would the AdSense algorithms respond in this situation?

Worse yet, now that the screen shot genie is out of the bag and everyone wants them on their sites for a variety of purposes, more and more screen shot providers are now taking more and more screen shots to fill this need. Before long a percentage of your daily ad impressions will probably be nothing more than screen shots.

It would be nice if the AdSense Advisor could chime in and let us know what impact, if any, these screen shots are having on our AdSense stats.

 

iwannano1




msg:3396195
 8:05 pm on Jul 16, 2007 (gmt 0)

Just block the bots using Apache web server, easy and simply.

incrediBILL




msg:3396229
 8:48 pm on Jul 16, 2007 (gmt 0)

Just block the bots using Apache web server, easy and simply.

I'm a rabid anti-scraper and bot blocker but that doesn't solve this problem as I allow ASK onto my site because it does send some traffic, it provides value. By blocking ASK you're cutting off the traffic it provides just to stop a few artificial AdSense page impressions which makes no sense. Besides, not all screen shot tools identify themselves as a BOT so it's not just a quick and easy "block them in Apache" solution.

If you don't get that ASK traffic you get fewer click thrus and a lower income.

The issue here isn't blocking the bot, that's a no-brainer if you wanted to do that and cut off the traffic they provide as well.

The issue is "Does Google count those page impressions from ASK, SNAP and others and do they impact your account?"

martinibuster




msg:3396246
 9:08 pm on Jul 16, 2007 (gmt 0)

Interesting Bill. I just did an Ask search on the name of a popular blog running AdSense across the top of the page. In the SERP click on the binoculars and you can see the snapshot, including an image of the AdSense unit.

timwestla




msg:3396253
 9:12 pm on Jul 16, 2007 (gmt 0)

The issue is "Does Google count those page impressions from ASK, SNAP and others and do they impact your account?"

Well you also mentioned that Ask made over 40,000 screen shots of your website. Did they show up in your AdSense reports? It seems like that many extra impressions would be pretty obvious, unless you're already getting a substantial number (like a million a day).

timwestla




msg:3396259
 9:22 pm on Jul 16, 2007 (gmt 0)

Both Ask and Snap make screen shots of various pages in your site...

I see what you're saying. I just searched for my domain name at Alexa.com and they do the same thing. There were a few dozen little snapshots of various pages, and I could see AdSense ads in each one of them.

The thing is, even if they took snapshots of all 200 pages of my site in one day, it wouldn't affect the statistics very much.

incrediBILL




msg:3396298
 9:58 pm on Jul 16, 2007 (gmt 0)

Ask made over 40,000 screen shots of your website... It seems like that many extra impressions would be pretty obvious

Actually that was Snap, ASK seems to be a little less aggressive.

Keep in mind that making screen shots is a slow process and they're making screen shots for many millions of websites. I get about 100K page impressions per day, not all are AdSense impressions though, but you can see it would be hard for me to easily tell the impact of these activities while a smaller site might notice right away. However, I've seen various IPs from Snap's screen shot tool [webmasterworld.com] ask for a couple of hundred pages a day for my site.

I never really thought this was a newsworthy item until I noticed more and more sites making screen shots and I figured if Google were actually filtering those sources (Ask, Snap, etc.) you should see PSA's on the screen shots and not real AdSense ads which clearly isn't the case.

As a matter of fact, I myself use a Windows-based tool called WebShot that uses MSIE to make screen shots. Using that tool I recently made about 30K screen shots for my directory site and they show AdSense if it's page. You wouldn't even know a screen shot had been taken because the WebShot tool drives MSIE to take the actual screen shots.

There are a lot of services that make thumb shots like WebSnapr and Thumbshots and these services have to update those images, as well as I'll have to update my images, so it's escalating a bit.

I wouldn't say that this activity is so large it's a problem for AdSense yet but I'm wondering if they're having any overall impact and what, if anything, Google is doing to filter out the obvious sources of this activity from our stats.

incrediBILL




msg:3396299
 10:01 pm on Jul 16, 2007 (gmt 0)

The thing is, even if they took snapshots of all 200 pages of my site in one day, it wouldn't affect the statistics very much.

Considering some people only get that much traffic total in a day, what would it do to THEIR sites?

I don't think large sites will have any issue, not until this becomes much more pervasive, but I'm curious about the small sites, could it cause smart pricing?

pontifex




msg:3396305
 10:04 pm on Jul 16, 2007 (gmt 0)

Before long a percentage of your daily ad impressions will probably be nothing more than screen shots.

Then I will send my robots to look at it :)

The question is, do they make a lot of screen shots, if your site is ranked high with them and it gives people an extra for their search terms, if they use these engines.

I doubt (out of the blue here), that they will take 40k screen shots of unimportant sites (unimportant along their algos).

P!

incrediBILL




msg:3396309
 10:11 pm on Jul 16, 2007 (gmt 0)

I doubt (out of the blue here), that they will take 40k screen shots of unimportant sites (unimportant along their algos)

Not sure about ASK yet (they've always crawled slow) but Snap went after every single page on my site.

They aren't the only ones doing screen shots either, just more commonly known.

timwestla




msg:3396317
 10:24 pm on Jul 16, 2007 (gmt 0)

Bill:

I never really thought this was a newsworthy item...

I think it's interesting :)

I wouldn't say that this activity is so large it's a problem for AdSense yet but I'm wondering if they're having any overall impact and what, if anything, Google is doing to filter out the obvious sources of this activity from our stats.

My guess is that it does impact the stats, but the effect is minimal and probably worth the extra vistors that the search engines bring to my site.

As for the smaller sites... it's possible that website traffic is taken into account. My guess is that low traffic sites aren't getting cached. As a test, I checked my employer's website. We don't focus at all on web traffic, and have done nothing to promote the site. We average a whopping 20 page views per day :) Our site didn't show up at all at ask.com, and at Alexa, the snapshot image is a placeholder that says "picture coming soon."

pontifex:

Then I will send my robots to look at it :)

LMAO!

Oops! Can I say "A" on here?

netmeg




msg:3396344
 11:11 pm on Jul 16, 2007 (gmt 0)

Yes, only here it's "laughing my adsense off"

zett




msg:3396610
 4:46 am on Jul 17, 2007 (gmt 0)

Interesting find. Which bots need to be blocked in order to get rid of the Ask network of sites? (I understand that they have at least two spiders, one for Ask and one for Snap.)

You see, I get virtually no traffic from this network (Ask, Excite, iWon or MyWay), so I guess I could easily cut this off.

jomaxx




msg:3396659
 6:06 am on Jul 17, 2007 (gmt 0)

I recall this coming up, probably here, a few months ago. At the time I was skeptical but then I did succeed in finding screenshots that showed actual AdSense ads. I would have to think that Google is on top of things by now.

incrediBILL




msg:3396665
 6:14 am on Jul 17, 2007 (gmt 0)

I would have to think that Google is on top of things by now.

You would think, but I'm seeing screen shots just days old with AdSense showing.

zett




msg:3396690
 6:37 am on Jul 17, 2007 (gmt 0)

As far as I understand this should only be a problem when Google is (a) counting these artificial ad impressions, and (b) when they use this for determing EPC or SmartPricing. I could imagine that this is not a serious problem to them. It can be, though, with regards to CPM ads and 40,000 pages being spidered... (In this case a lot of invalid views would be generated by those bots.)

martinibuster




msg:3396725
 7:39 am on Jul 17, 2007 (gmt 0)

zett, you've hit on a good reason why this is important. If AdSense is charging advertisers for impressions by bots from Ask, Snap, and Alexa then Google may owe CPM advertisers a refund.

incrediBILL




msg:3396735
 7:46 am on Jul 17, 2007 (gmt 0)

charging advertisers for impressions by bots from Ask, Snap, and Alexa then Google may owe CPM advertisers a refund

BINGO!

I hate to give away all the answers but CPM ads tend to repeat often on some sites and could be a significant part of 40K impressions on the same site, especially if you have multiple ad units, a cool 1K of wasted impressions to a single CPM campaign are entirely possible.

Kurgano




msg:3396746
 8:09 am on Jul 17, 2007 (gmt 0)

Adsense ads continue to show in cached copies of your site even six months after it no longer exists. The wayback machine contains more than just a screenshot, it contains the code as it used to be. Images return a red x because they no longer exist but the adsense code carries on.

m0thman




msg:3396750
 8:18 am on Jul 17, 2007 (gmt 0)

Would have thought that Google could determine whether or not a bot or a bod was calling the javascript and respond accordingly. Far easier than half the webmasters on the planet coding a response. Fair point inflated CPM for advertisers by the way, that can't be good for anyone.

vincevincevince




msg:3396757
 8:28 am on Jul 17, 2007 (gmt 0)

If there are ads shown in those screenshots then the services creating them are flouting the robots.txt guidelines:

[pagead2.googlesyndication.com...]
User-Agent: *
Disallow: /
Noindex: /

Tastatura




msg:3396777
 8:54 am on Jul 17, 2007 (gmt 0)

If there are ads shown in those screenshots then the services creating them are flouting the robots.txt guidelines:

[pagead2.googlesyndication.com...]
User-Agent: *
Disallow: /
Noindex: /

interesting point, but I am not quite sure I agree because "imaging" bots are not directly calling (going to) G syndication domain - they call one of our pages (URI to be precise) and execute javascript on it, and javascript on our pages calls adsense delivery (my description is clear as mud but hope it makes sense :) )

[edited by: Tastatura at 8:54 am (utc) on July 17, 2007]

vincevincevince




msg:3396784
 9:04 am on Jul 17, 2007 (gmt 0)

Tastatura,

The javascript parsing or other imaging is still part of that bot's software and should be covered under the same robots.txt rules.

[edit]The answer is obviously to get the bots to click the ads, in order to ensure that our statistics aren't messed up...[/edit]

[edited by: vincevincevince at 9:12 am (utc) on July 17, 2007]

MikeNoLastName




msg:3396790
 9:11 am on Jul 17, 2007 (gmt 0)

>"I doubt (out of the blue here), that they will take 40k screen shots of unimportant sites (unimportant along their algos). "
"As far as I understand this should only be a problem when Google is (b) when they use this for determing EPC or SmartPricing."

But if EVERYONE is seeing the same activity the smart pricing should even out over the long run, across all accounts, and not have any individual impact.

[edited by: MikeNoLastName at 9:13 am (utc) on July 17, 2007]

Rosalind




msg:3396905
 12:01 pm on Jul 17, 2007 (gmt 0)

But if EVERYONE is seeing the same activity the smart pricing should even out over the long run, across all accounts, and not have any individual impact.

Except that some will ban the bots, and some won't.

AdSenseAdvisor




msg:3396956
 1:15 pm on Jul 17, 2007 (gmt 0)

I'm actually out of the US this week (not an excuse - I know you int'l publishers deal with AdSense time differences every day), but I'm going to try to get in touch with my team and see if I can get more information about this issue.

-ASA

amznVibe




msg:3397057
 2:47 pm on Jul 17, 2007 (gmt 0)

Why would a screenshot be taken of more than your front page?

Why does ASK need a screenshot of every single page? Weirdness.

Personally I love thumbshots of the site (front page) I am about to visit, it really enhances the search experience - example:
xs217.xs.to/xs217/07291/adwords-graphic-128.png

explorador




msg:3397106
 3:13 pm on Jul 17, 2007 (gmt 0)

That means your AdSense ads are also being included in those screen shots and possibly skewing your statistics.

I'm there. My sites have more visits that I know they really have. There are tons of hits by non-visitors (bots). I had to block a lot of things as some areas consume cpu resources.

Just block the bots using Apache web server, easy and simply.

Is not that easy... I had (still have) problems with bots that do nothing but consume resources. I blocked some and some still come back (ip deny and robots.txt). htacess is something I still don't quite manage to risk traffic. Also, every month a new bot appears from whatever .edu "for research purposes".

There is a lot to say about blocking bots and the drawbacks on using htaccess.

JohnKelly




msg:3397134
 3:51 pm on Jul 17, 2007 (gmt 0)

If your site gets traffic from Ask or Snap but you want to block the AdSense impressions from them, just do it via a PHP (or ASP) script.

All my AdSense ads are displayed in a little PHP include file. I have a flag set to not show AdSense ads if visiting via certain ISP's (such as my own), to prevent accidental clicks. You could simply add the crawler's hostname or IP address to the list. That way, your page would be crawled but not the AdSense block.

carguy84




msg:3397282
 6:05 pm on Jul 17, 2007 (gmt 0)

^^^ that's what I was going to suggest, just don't show the ads to the bots.

This 46 message thread spans 2 pages: 46 ( [1] 2 > >
Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google AdSense
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved