Welcome to WebmasterWorld Guest from 220.127.116.11
Question is -- what's a valid sampling size for adsense? How many impressions and/or days should you test until you can confidently base decisions on the data?
And sure, this will always be dependent on what niche the site is in. But figuring some numbers/percentages that could hold up to mild scientific scrutiny should be possible, yes?
As with most things relating to AdSense, there is no straight forward answer. It depends..
..if you have got the tags on all the pges you are ever going to put them on, and your site is already doing as well as you expect in Google serps, then about a week gives you a good idea...
...provided that you are getting enough clicks. I reckon you have to look at blocks of 100 clicks to see any valid movements
...and blocks of perhaps 2000 impressions
If you have a reasonable sized site then you will be tweaking banners/skyscrapers, page placement, keyword targeting, ....in which case you will need more time to see where you are going.
Personally I would give it a month before you take any major long term decisions
First, what exactly are you trying to measure? Click-thrus to sales, etc. This will dictate how confident you should be that the results are indeed true and that tells you how many samples you need. Of course costs in time, etc. usually dictates how many samples one takes, whether that sample size is just or not.
There are 2 different kinds of data in statistics. Variable, which is like measuring the diameter of a circle, where you get a real number and attribute, which is like a go-no go gauge or how many widgets are totally painted vs ones that lack paint in certain areas. You are measuring the latter.
According to Juran, a student of Walter A. Shewhart, you need at least 125 samples for variable data. Motorola did a study not so long ago where they apparently measured confidence levels of attribute data using variable data sample sizes. Their conclusion was, to have the same confidence level with attribute data, you need at least twice the variable data sample size.
Typically, the more data you have, the higher your confidence level should be, providing you follow and know all the other technical stuff about statistics. You see sample size is a science all in itself under the statistical umbrella.
Hope this helps.