homepage Welcome to WebmasterWorld Guest from 54.204.168.212
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member
Home / Forums Index / Google / Google AdWords
Forum Library, Charter, Moderators: buckworks & eWhisper & skibum

Google AdWords Forum

    
Profit by Impression / Profit by Click
Two measurements for successful split testing
eWhisper

WebmasterWorld Administrator ewhisper us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 7201 posted 3:17 pm on Jan 8, 2006 (gmt 0)

The current success measurements for PPC Split Testing often do not take into account every factor that needs to be determined. Each metric only gives a portion of the story, they do not actually tell you the entire story.

At the last Pubcon [webmasterworld.com], I briefly mentioned two concepts I've been using since 2002 that attempt to bring more factors into play when measuring split testing results: profit by impression and profit by click. When I mentioned these metrics, I was greeted with a room full of blank stares. Later, I received several emails asking for the concept to be spelled out. So here it is, a brief overview of Profit by Impression (PBI) & Profit by Click (PBC).

Please Note:

When considering metrics for split testing, these are three measurements that truly matter:

  • Profit
  • Profit by Impressions
  • Profit by Click

Profit is a metric for an entire campaign (more below).
Profit by click/impression should be the success metrics for split testing.

In split testing [webmasterworld.com] one wants to determine what ad copy, landing page, keywords, etc are providing the best value for your site. The reason profit alone does not matter with split testing is that often all of your tests do not receive the same exposure, therefore your metrics could be skewed by a single ad or landing page receiving too much exposure, receiving a high ROI, but poor CTR, etc. In split testing you are learning about your advertising - not making final decisions.

The entire conversion process starts at the search query. Not the click. Not the page view. Not the sale. In order to understand the entire chain of search query to ad copy view to click to landing page to conversion one has to use two new metrics: Profit by Impression (PBI) & Profit by Click (PBC).

Consider the current success metrics:

CTR (click through rate) only measures how many people click on your ad. It does not measure conversion rate or profits.

Average CPC (cost per click) only measures what you pay for a click, again, there is no conversion data associated with this metric.

Conversion rate is a percentage of how many people converted compared to saw your landing page. It does not include CTR, CPC (cost per click), ad spend or even profit margin.

Cost per conversion tells you how much it cost you to have a visitor perform a specific action. It does not include how many people converted, the conversion percentage, or even the ad's click through rate.

ROI (Return on Investment) / ROAS (Return on Ad Spend)-These are percentage metrics of profit compared to costs. They do not incorporate click through rate, conversion percentage, or ad views.

Profit should be a metric that looks at a campaign holistically. If you want to look at individual parts of a campaign, you should also look at individual parts of the profit. (If looking at an entire campaign, please read: Forget ROI: Show me the Profits [webmasterworld.com])

Each of the above metrics is just that, a metric. One piece of information in a sea of statistics.

The PPC Process
If you are familiar with PPC, then you should be intimately familiar with the search process.

Brief recap (from PPC Metrics [webmasterworld.com]):

  • PPC starts at the search query (searcher formulates idea)
  • Your cost starts at the click (searched connects with your ad and clicks on it).
  • If landing page meets expectations and convinces someone to buy from you, then you have a conversion.
  • Search Query > Click > Landing Page > Conversion

Profit by Click / Impression

Since the conversion cycle starts with the impression, this is where the calculation of statistics should also begin.

Let's consider the below chart. This chart is based upon a fixed fee $25 per sale. It's also based on testing 5 (A-E) different scenarios.

A 'scenario' is a set of values you are testing. This could be:

  • 5 different tag lines
  • 5 different titles
  • 5 completely different ads
  • 5 different ads going to 5 different landing pages
  • 1 ad going to 5 different landing pages
  • etc.

Don't feel limited by what you can test. You can test anything. It's a matter of understanding the metrics after you've run a test that is important to learn.

.............TestA ...TestB ...TestC ...TestD ...TestE
Impressions .6703 ....9080 ....7858 ....8956 ....5689
Clicks ......200 .....450 .....420 .....896 .....256
CTR .........2.98% ...4.96% ...5.34% ...10.00% ..4.50%
Avg CPC .....$1.06 ...$0.35 ...$0.52 ...$0.63 ...$3.99
Cost ........$212.56 .$156.32 .$220.25 .$560.32 .$1,020.56
Conversions ..32 .....34 ......44 ......58 ......59
Conversion % .16.00% .7.56% ...10.48% ..6.47% ...23.05%
Cost per
Conversion ..$6.64 ..$4.60 ....$5.01 ...$9.66 ...$17.30
Sales($25/
conversion) $800.00 ..$850.00 .$1,100.00 $1,450.00 $1,475.00
Profit .....$587.44 ..$693.68 .$879.75 ..$889.68 $454.44
ROI ........376.36% ..543.76% .499.43% .258.78% 144.53%
Profit per
Click ......$2.94 ....$1.54 ..$2.09 ....$0.99 ...$1.78
Profit per
Impression .$0.09 ....$0.08 ..$0.11 ....$0.10 ...$0.08

Highlighted in blue are the top performing rows for each statistic.

In this chart, there are two obvious winners:

  • Test A for profit by click
  • Test C for profit by impression

It is important to note, neither of these columns are the tops in terms of CTR, Sales, ROI, Conversion Rate or Cost per Conversion.

Why did they win this test if they aren't tops for any of those statistics?

Simple, those statistics only tell part of the story. Test B has the lowest cost per conversion, the highest ROI, the lowest average CPC. In most scenarios, people would consider that a winner. However, when we look at how much each of the tests make per click (per visit) this test doesn't win out because of how the entire conversion process takes place: search query to click to landing page to conversion.

Test C is a clear winner in profit by impression.

It is also important to note, if the above numbers were entire campaigns and not a split test (i.e. the results were a campaign in it's entirety being measured by it's performance) then Test D would be a clear winner due to it's profit margin.

If we received the above results on a test, what would be the next step?

There are a few answers:

  1. Analyze Test A to see if there is a way to bring in more clicks.
  2. Analyze Test C to see if there is a way to bring in more impressions.
  3. Analyze the statistical winner to learn why they won that metric.
    • Can Test B tell us how to get more impressions?
    • Can Test E tell us how to convert visitors?
    • Can Test D tell us how to get more clicks?
    • Etc…

  4. Put together a strategy for A & C based on the numbers (let the numbers tell the story. Theory is great, but eventually, numbers will give you facts)
  5. Repeat test.
  6. Measure new results.
  7. Repeat.

When running a test, one has to both understand when a variable is the best to reuse, but also why certain variables won individual metrics (i.e. why an ad has a high CTR, why a landing page has a high conversion rate, etc). By combining the Profit By Click and Profit By Impression winners with the knowledge of why the other variables preformed best in specific metrics then one can leverage the entire testing statistics in putting together new ads, landing pages, and tests.

The above test was run with a static item cost ($25). For many sites, the total order and average order amount will also be statistics you wish to measure.

In the below chart, the base numbers are the same, the only number that was changed was Sales $$. This is the combined value of all the sales made during the test period. Of course, if the sales value is changed, the profit margin will also be affected. Average sale was added as a metric.


.............TestA ...TestB ...TestC ...TestD ...TestE
Impressions .6703 ....9080 ....7858 ....8956 ....5689
Clicks ......200 .....450 .....420 .....896 .....256
CTR ........2.98% ...4.96% ...5.34% ...10.00% ..4.50%
Avg CPC ....$1.06 ...$0.35 ...$0.52 ...$0.63 ...$3.99
Cost .......$212.56 .$156.32 .$220.25 .$560.32 .$1,020.56
Conversions .32 ......34 ......44 ......58 ......59
Conversion
Rate .......16.00% ..7.56% ...10.48% ...6.47% ...23.05%
Cost per
Conversion .$6.64 ...$4.60 ...$5.01 ...$9.66 ...$17.30
Sales $$ ...$560.23 .$200.23 .$567.23 .$1,420.00 $1,489.12
Avg Sale ...$17.51 ..$5.89 ...$12.89 ..$24.48 ...$25.24
Profit .....$347.67 .$43.91 ..$346.98 .$859.68 ..$468.56
ROI ........263.56% .128.09% .257.54% .253.43% ..145.91%
Profit per
Click ......$1.74 ...$0.10 ..$0.83 ...$0.96 ....$1.83
Profit per
Impression ..$0.05 ..$0.00 ..$0.04 ...$0.10 .....$0.08

The above numbers can be analyzed just like the first chart above. It's important to show how average sale and total sales can affect the test. The two previous winners are no longer the top performing tests. Test D, which only won on CTR earlier, is the clear Profit by Impression winner. Test E is the clear Profit by Click Winner.

In this scenario, besides the information that we already walked through that needs to be analyzed, there is an emerging trend. Test E is full of converting traffic (which could be the keywords, ad copy that speaks only to buyers, etc). Determining why it has such a great conversion rate and then applying those results to Test D (which has the higher CTR, 3rd lowest CPC, 2nd highest conversions, 2nd highest sales, 2nd highest avg sale, highest profit) would bring about some pretty significant changes and improvements.

These tests are but two simple scenarios in a sea of online metrics. One could easily add page views per visitor, time spent on site per keyword, Value of a Lifetime Customer, catalog requests, etc. Metrics are easy to add, what's important is assigning a dollar amount to those metrics. Determine what a phone call is worth, what a catalog request brings in sales, what an email does for your bottom line. Determine not only what action you want someone to take - but what that value is to your company.

Conclusion

Profit by impression (PBI) and Profit by Click (PBC) are the best metrics to determine when split testing ad copy or landing pages. One can determine profit by impression/click numbers by individual line of an ad copy, by the entire ad copy, by landing page, etc.

When split testing, think about the actual goal: user action. By understanding how each line of an ad interacts with a landing page and with a search query, by understanding how a landing page is affected by a keyword, by understanding how a keyword interacts with a specific ad, by understanding all of your metrics you can finally determine which is the proper combination to be running.

If you know these statistics:

  • Impressions
  • Clicks
  • Cost of ad campaign
  • Conversions
  • Sales $$

You can derive these statistics:

  • Click through rate (CTR)
  • Average cost per click (CPC)
  • Conversion rate
  • Cost per conversion
  • Average sale
  • Profit
  • ROI
  • ROAS
  • Profit by Click
  • Profit by Impression

These are elements in a PPC campaign that you control. If you can measure it and understand the metric, you can change it. Knowledge of metrics is key. Knowledge of what to do with the metrics is crucial.

Profit by Impression & Profit by Click are still underutilized actions in the world of PPC; however, they are concepts that have been successfully implemented for several years.

Only by understanding how the entire PPC cycle works, from query to ad copy to click to landing page to user action can one really understand the value of advertising.

Understand the value. Measure the value. Improve the value.

Three simple steps to successful advertising.

Related threads:
Split testing Ad Copy & Landing Pages: [webmasterworld.com...]
A/B Testing (supporters) [webmasterworld.com...]

 

odddogatwork

5+ Year Member



 
Msg#: 7201 posted 9:59 am on Jan 9, 2006 (gmt 0)

thank you.

of to do some pre test preparation ...

Sports Workout

10+ Year Member



 
Msg#: 7201 posted 10:30 am on Jan 9, 2006 (gmt 0)

Badda Bing...yet another eWhisper post "flagged" down.

bts111

10+ Year Member



 
Msg#: 7201 posted 11:47 am on Jan 9, 2006 (gmt 0)

Thank you professor :)

marketingagency

5+ Year Member



 
Msg#: 7201 posted 12:15 pm on Jan 9, 2006 (gmt 0)

This is real good analysis. I do work similar grounds but working on different avg sales [I am assuming it to be a group of products with varying price tags].

eWhisper Said: -0-
[The above numbers can be analyzed just like the first chart above. It's important to show how average sale and total sales can affect the test.]
eWhisper Said: -0-

Adding to your statement, here is something to show how important is Average sales.

............TestA1...TestA2...TestA3.....TestA4......TestA5
Impressions..6703....6703.....6703.......6703........6703
Clicks.......200......200......200........200........200
CTR.........2.98%.....2.98%....2.98%......2.98%......2.98%
Avg CPC......$1.06....$1.06.....$1.06 ....$1.06..........$1.06
Cost.....$212.56....$212.56..$212.56..$212.56........$212.56
Conversions.32.....32........32.......32.............32
Conversion%.16.00%.16.00%....16.00%...16.00%.........16.00%
Cost per
Conversion.$6.64..$6.64......$6.64....$6.64..........$6.64
Sales$$....$560.23..$200.23..$567.23..$1,420.00.....1,489.12
Avg Sale..$17.51...$5.89....$12.89...$24.48..............$25.24
Profit..$347.67.(-$12.33).$354.67..$1,207.44........$1,276.56
ROI........263.56%..94.20%.....266.86%....668.05.....700.56%
Profit Per
Click.......$1.73..(-$0.06).....$1.77....$6.02.......$6.37
Profit
Per
Impression..$0.05...($0.00)......$0.05..$0.18........$0.19

It has always been the case that the Avg Value of the Products hits the profit.

Very Well done eWhisper!

MarketingAgency

arinick

5+ Year Member



 
Msg#: 7201 posted 4:23 pm on Jan 9, 2006 (gmt 0)

From an agency perspective, this creative approach to campaign analysis is invaluable. For those of us that charge management fees based on media spend, helping clients understand their profit per impression (assuming that there is a positive profit per impression) can only lead to one question --- "So, how many impressions can I get?". Great stuff, eWhisper.

cerebrum

5+ Year Member



 
Msg#: 7201 posted 5:10 pm on Jan 9, 2006 (gmt 0)

Excellent Post. Thank you for sharing with us.

marketingmagic

5+ Year Member



 
Msg#: 7201 posted 6:41 pm on Jan 9, 2006 (gmt 0)

Yes, excellent post, very useful info! :-)

Essex_boy

WebmasterWorld Senior Member essex_boy us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 7201 posted 8:58 pm on Jan 9, 2006 (gmt 0)

Ive never heard of anyone on here getting a 6% + conversion rate.

Any one say they have?

arinick

5+ Year Member



 
Msg#: 7201 posted 10:08 pm on Jan 9, 2006 (gmt 0)

I have clients who get over 10% conversion rate in competitive industries. What's more-- I've even got some that do so from the Content network. I know of other people who have clients who do even better. Just depends on your space in your industry.

Marty_Foley

10+ Year Member



 
Msg#: 7201 posted 11:05 pm on Jan 9, 2006 (gmt 0)

Question for eWhisper:

Have you ever investigated the use of multi-variate testing of Adwords, based on the "Taguchi Method"?

It's a proven method let you test literally dozens, hundreds, or even thousands of different combinations of ad variables at once, with a small number of ads.

As one example of its effectiveness, it has helped me go from a few hundred dollars a month, to around $3,000.00 to $4,000.00 a month, just for one affiliate program.

marketingagency

5+ Year Member



 
Msg#: 7201 posted 7:47 am on Jan 10, 2006 (gmt 0)

Arnick. I have one thing to add to the agency approach towards these numbers. PPC has given the right path to understand how the media budget is spend on paid search media and any other media. Agency method will ultimately have the bottle neck i.e. conversions.

I have clients where the conversion% is more than 12%. Again, supporting to what eWhisper's TEST, Avg Sale is the Key to the profit from any campaign.

MarketingAgency

eWhisper

WebmasterWorld Administrator ewhisper us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 7201 posted 2:02 pm on Jan 10, 2006 (gmt 0)

Have you ever investigated the use of multi-variate testing of Adwords, based on the "Taguchi Method"?

First, one has to agree with the methodology. The Taguchi method assumes each variable is independent in the test environment. If you believe there is an interaction between the ads and landing page, or if you believe that various parts of the landing page interact with each other, then the methodology is flawed.

However, I have seen people do well with the method as it was the first testing they ever employed. The sheer act of doing any testing, by any methodology, was most likely going to improve their results, and because they employed this method, they attributed their success to it, when there were other tests which would have also shown similar results.

For those unfamiliar with the word, Taguchi, it is also known as the Robust Design method developed by Dr. Genichi Taguchi. It is a methodology for testing engineering productivity by treating every variable as independent. That is the crux of the issue, the variables as independent. If you believe that ads and landing pages (or parts of an ad or part of a landing page) are interlaced, then the method doesn't address that testing (outside of doing any testing at all -and any test will start to show you some information about your campaign).

There are those who only test the ads with the Taguchi method, or only test the landing pages. However, there is a landing page and ad interaction that happens as well, and I've never seen someone do a test where there are two different elements (ads and landing page) combined into a single Taguchi test (if anyone has a case study displaying that methodology - I'd love to read it).

I've seen the Taguchi method employed well for One Page Wonder [webmasterworld.com] sites as their structure can often be moved around based on the confidence factors. The elements are not necessarily as interlaced as a website, and thus for one page wonders, this method can work very well.

I've seen the method work well for split testing ads when one is looking for a single variable in the results.

I've also seen people who said they used the Taguchi test, when in fact, they were running several split tests at once (not saying that's what you did - just my observations of what I've seen).

I've not seen someone who has successfully used the Taguchi method to measure all aspects of an ad, landing page, and their interaction within a single test.

The Taguchi math is sound. The implementation for certain tests is also sound. The crux from what I've seen is if you believe that the elements don't interact with each other as they're considered indecent variables by the Taguchi method.

If someone has successfully implemented a Taguchi test for ads and landing pages at a single time, I'd love to see a methodology post on how the test was run.

[added]Just one last thought I had on the method. The Taguchi method depends on a lot of traffic. Because it's using statistical averages for element confidence, the results on a small amount of traffic can be misleading. This makes it so there are some who have employed this method on not enough traffic and actually lowered their results as they didn't put enough traffic through the test to come up with the actual averages.

Of course, it's possible that the average happens immediately. It's also possible to see variances in traffic (hence why ads can do well one week, poor the next, great for the next 4 - it's about averages over time based on the traffic). Hence, the method is dangerous to use if you don't put enough traffic through the testing phase before you make adjustments based on your results[/added]

sharewarepro

10+ Year Member



 
Msg#: 7201 posted 3:08 pm on Jan 10, 2006 (gmt 0)

" Ive never heard of anyone on here getting a 6% +conversion rate."

Yes, definitely!

Dave Collins
SharewarePromotions

Essex_boy

WebmasterWorld Senior Member essex_boy us a WebmasterWorld Top Contributor of All Time 10+ Year Member



 
Msg#: 7201 posted 9:16 pm on Jan 10, 2006 (gmt 0)

Guess I should get out more then...

sharewarepro

10+ Year Member



 
Msg#: 7201 posted 8:36 am on Jan 11, 2006 (gmt 0)

Or less!

Dave Collins
SharewarePromotions

percentages

WebmasterWorld Senior Member 10+ Year Member



 
Msg#: 7201 posted 9:48 am on Jan 11, 2006 (gmt 0)

>I've never heard of anyone on here getting a 6% + conversion rate.

Well you just haven't lived yet!

For me, 98+% is the norm....and not hard to achieve.

>When split testing, think about the actual goal: user action.

I loved your post (a tad long, but enjoyable). But, User Action is actually the only thing that matters. Maybe selling something that costs nothing is the way to go? I know that 99%+ of you will find that confusing.......it was only intended for the other 1%.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google AdWords
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved