Testing for Ad Effectiveness

Banner ads

[tmadel]

Are there any benchmarks on the number of impressions you need to have to determine the effectiveness of a banner ad?

We have a number of different (and disparate) websites. We are selling banner ads on these sites, but we have some excess inventory and would like to cross-promote our sites on each other.

How many impressions/CT do you need to get a statistically valid sample and determine if the ad is effective on each site? Are there any general guidelines?

[eljefe3]

Welcome to WebmasterWorld. Not sure if there is one benchmark, but I wouldn't make any observations on less than 100,000 impressions, but 1 million would be even better.

[markwelch]

It is true that you usually don't have "statistical significance" within a narrow margin unless you have at least 100,000 adviews.

However, I generally start making preliminary decisions after 1,000 to 10,000 adviews, if I'm paying for campaigns on a CPM basis. Thus, if I've paid $5 CPM for 10,000 adviews ($50) and drawn only 1 or 2 clicks, I'm likely to suspend that campaign.

If I am running A-B tests on Google AdWords for PPC advertising, I generally make decisions about "which creative" (which ad text or which image) or "which landing page" after 25,000 to 100,000 adviews per creative, but if there is a dramatic difference after 10,000 adviews I might make the decision then.