A/B/C testing conclusion

Large rectangle versus leaderboard versus wide skyscraper.

[UserFriendly]

I wanted to test the performance of three different ad formats on one section of my site. Using a randomizer, I had three different formats tested on several pages on my site:

Large rectangle (336x280);
Leaderboard (728x90);
Wide skyscraper (160x600).

One of the three formats was selected randomly each time, to appear immediately before the body text of the article on the page. That is, after the navigation, page title and page introduction. The page width was such that the leaderboard would fill the width of the body area. The large rectangle and wide skyscraper were made to float on the left, so that text flowed around them. All three were blended with the colours of the page, with their background and border colours the same as the page background.

The ads shown in each format are seemingly the same, though occassionally may appear in a different order, or an ad may appear only in one format.

My site is low traffic (compared to the big guns on this forum), but after two months each ad format has seen a decent number of impressions, and a non-trivial number of clicks.

Without wanting to break AdSense terms and conditions, I'll summarise the result as best I can.

Large rectangle received the least impressions (as the randomizer is random and not done on a rotary basis), having 94% the number of impressions that the most-impressed format received.

Regardless, the large rectangle saw the highest numbers in all other areas. Its all-time CTR is 31% higher than the leaderboard, and 56% higher than the wide skyscraper.

The large rectangle has an eCPM 42% higher than the leaderboard's, and 250% higher than the eCPM that the wide skyscraper returned.

Which all means that the large rectangle produced the highest earnings for the test period, generating 39% more revenue than the leaderboard, and 230% more revenue than the wide skyscraper.

So I'm now turning off the randomizer and replacing the random code with the AdSense large rectangle code. Please note that I'm not saying that the leaderboard and wide skyscraper don't have their uses. Just that at the top of these pages, on my site, the large rectangle seems to be the best performer.

Has anyone else (preferably with better traffic figures than I see) done a similar test?

[joeking]

thanks for taking the time to post this userfriendly

[david_uk]

Historicall the large rectangle has always performed well for me, but I'm currently trying some a/b testing between the normal rectangle and wide skyscraper on one page. Over the last month the impressions have been pretty well equal, as have the earnings. The wide skyscraper has twice the ctr but half the epc, and the rectangle has twice the epc but half the ctr!

But to add a further slant to this, I actually think that the a/b testing itself is increasing the figures! It seems to me that the rotation of the ad blocks has increased overall ctr and earnings. Therefore I think the randomised testing is likely to be a permanent fixture.

[Pedent]

A very useful post; well-conceived and -explained.

I agree with david_uk, though: no ad will perform as well when it's getting 100% of the impressions as when it's being rotated with other formats.

[trannack]

This is what this forum should be all about. The sharing and pooling of information. thanks guys for your input. For me I have always found that the large rectangle works best. I have tried various other formats, but I always end up back at quare one. Also i find that the structure of the lg rectangle works much better on my pages.

[GoldenHammer]

Yes, the test itself may just make a difference, I wonder if you would receive a similar earning by the big rectangle only.

I am going to try that on some of my higher traffic pages, the rotator may makes even significant different.

[UserFriendly]

Yes, I did wonder whether the randomization itself was affecting the total earnings of the pages. I'll have to report back in a couple of months' time, let you know how the non-random performance was.

[bbd2000]

Great post and very timely for me, I’m in the middle of building a new site and redoing another one. I have always preferred Leaderboards over the Wide skyscraper. In my experience the Leaderboards always beat the skyscraper in terms of income. I rarely used the Large rectangle because it just didn’t fit the layout of my sites and I wanted to avoid the “in you face” look of them. However, since I have been planning my redesign, I have looked at hundreds of sights searching for layouts that I like. My conclusion is that there is definitely a place for the Large rectangle’s if integrated into the design from the beginning. Now that you have conducted this experiment I think I will give them a try.

[UserFriendly]

I agree. There's no point shoe-horning a format in just because it works for someone else.

But on my pages, the large rectangle and leaderboard looked right. The wide skyscraper fit nicely, but didn't look as at-home as the other two.