|What's a Valid Sample Size|
CTR, CPC, blah blah blah blah blah
So lets say you want to test out AdSense and get a real feel for what it can or can't do.
Question is -- what's a valid sampling size for adsense? How many impressions and/or days should you test until you can confidently base decisions on the data?
And sure, this will always be dependent on what niche the site is in. But figuring some numbers/percentages that could hold up to mild scientific scrutiny should be possible, yes?
Looking at the variability I've seen, you're talking at least two weeks to get a basic idea and a few months before you will be able to predict with confidence.
>> How many impressions and/or days should you test until you can confidently base decisions on the data
As with most things relating to AdSense, there is no straight forward answer. It depends..
..if you have got the tags on all the pges you are ever going to put them on, and your site is already doing as well as you expect in Google serps, then about a week gives you a good idea...
...provided that you are getting enough clicks. I reckon you have to look at blocks of 100 clicks to see any valid movements
...and blocks of perhaps 2000 impressions
If you have a reasonable sized site then you will be tweaking banners/skyscrapers, page placement, keyword targeting, ....in which case you will need more time to see where you are going.
Personally I would give it a month before you take any major long term decisions
I have been doing a whole lot of tweaking on my sites. Made many placement changes, removed several affiliate links, relocated some affiliate links, et al so that more focus is provided to the skyscraper, and everytime I notice better and better results. My final take is to keep the page completely uncluttered with a focus on the content & the sykscraper. Of course, it took around 20-25 days to do all these testing.
>what's a valid sampling size for adsense?
We do statistics for food, and have been doing statistics for 20 years. So this is real technical and I donít know how exact you want to be.
First, what exactly are you trying to measure? Click-thrus to sales, etc. This will dictate how confident you should be that the results are indeed true and that tells you how many samples you need. Of course costs in time, etc. usually dictates how many samples one takes, whether that sample size is just or not.
There are 2 different kinds of data in statistics. Variable, which is like measuring the diameter of a circle, where you get a real number and attribute, which is like a go-no go gauge or how many widgets are totally painted vs ones that lack paint in certain areas. You are measuring the latter.
According to Juran, a student of Walter A. Shewhart, you need at least 125 samples for variable data. Motorola did a study not so long ago where they apparently measured confidence levels of attribute data using variable data sample sizes. Their conclusion was, to have the same confidence level with attribute data, you need at least twice the variable data sample size.
Typically, the more data you have, the higher your confidence level should be, providing you follow and know all the other technical stuff about statistics. You see sample size is a science all in itself under the statistical umbrella.
Hope this helps.