eWhisper - 3:17 pm on Jan 8, 2006 (gmt 0) At the last Pubcon [webmasterworld.com], I briefly mentioned two concepts I've been using since 2002 that attempt to bring more factors into play when measuring split testing results: profit by impression and profit by click. When I mentioned these metrics, I was greeted with a room full of blank stares. Later, I received several emails asking for the concept to be spelled out. So here it is, a brief overview of Profit by Impression (PBI) & Profit by Click (PBC). Please Note:
The current success measurements for PPC Split Testing often do not take into account every factor that needs to be determined. Each metric only gives a portion of the story, they do not actually tell you the entire story.
At the last Pubcon [webmasterworld.com], I briefly mentioned two concepts I've been using since 2002 that attempt to bring more factors into play when measuring split testing results: profit by impression and profit by click. When I mentioned these metrics, I was greeted with a room full of blank stares. Later, I received several emails asking for the concept to be spelled out. So here it is, a brief overview of Profit by Impression (PBI) & Profit by Click (PBC).
When considering metrics for split testing, these are three measurements that truly matter:
Profit is a metric for an entire campaign (more below).
Profit by click/impression should be the success metrics for split testing.
In split testing [webmasterworld.com] one wants to determine what ad copy, landing page, keywords, etc are providing the best value for your site. The reason profit alone does not matter with split testing is that often all of your tests do not receive the same exposure, therefore your metrics could be skewed by a single ad or landing page receiving too much exposure, receiving a high ROI, but poor CTR, etc. In split testing you are learning about your advertising - not making final decisions.
The entire conversion process starts at the search query. Not the click. Not the page view. Not the sale. In order to understand the entire chain of search query to ad copy view to click to landing page to conversion one has to use two new metrics: Profit by Impression (PBI) & Profit by Click (PBC).
Consider the current success metrics:
CTR (click through rate) only measures how many people click on your ad. It does not measure conversion rate or profits.
Average CPC (cost per click) only measures what you pay for a click, again, there is no conversion data associated with this metric.
Conversion rate is a percentage of how many people converted compared to saw your landing page. It does not include CTR, CPC (cost per click), ad spend or even profit margin.
Cost per conversion tells you how much it cost you to have a visitor perform a specific action. It does not include how many people converted, the conversion percentage, or even the ad's click through rate.
ROI (Return on Investment) / ROAS (Return on Ad Spend)-These are percentage metrics of profit compared to costs. They do not incorporate click through rate, conversion percentage, or ad views.
Profit should be a metric that looks at a campaign holistically. If you want to look at individual parts of a campaign, you should also look at individual parts of the profit. (If looking at an entire campaign, please read: Forget ROI: Show me the Profits [webmasterworld.com])
Each of the above metrics is just that, a metric. One piece of information in a sea of statistics.
The PPC Process
If you are familiar with PPC, then you should be intimately familiar with the search process.
Brief recap (from PPC Metrics [webmasterworld.com]):
Profit by Click / Impression
Since the conversion cycle starts with the impression, this is where the calculation of statistics should also begin.
Let's consider the below chart. This chart is based upon a fixed fee $25 per sale. It's also based on testing 5 (A-E) different scenarios.
A 'scenario' is a set of values you are testing. This could be:
Don't feel limited by what you can test. You can test anything. It's a matter of understanding the metrics after you've run a test that is important to learn.
Highlighted in blue are the top performing rows for each statistic.
In this chart, there are two obvious winners:
It is important to note, neither of these columns are the tops in terms of CTR, Sales, ROI, Conversion Rate or Cost per Conversion.
Why did they win this test if they aren't tops for any of those statistics?
Simple, those statistics only tell part of the story. Test B has the lowest cost per conversion, the highest ROI, the lowest average CPC. In most scenarios, people would consider that a winner. However, when we look at how much each of the tests make per click (per visit) this test doesn't win out because of how the entire conversion process takes place: search query to click to landing page to conversion.
Test C is a clear winner in profit by impression.
It is also important to note, if the above numbers were entire campaigns and not a split test (i.e. the results were a campaign in it's entirety being measured by it's performance) then Test D would be a clear winner due to it's profit margin.
If we received the above results on a test, what would be the next step?
There are a few answers:
When running a test, one has to both understand when a variable is the best to reuse, but also why certain variables won individual metrics (i.e. why an ad has a high CTR, why a landing page has a high conversion rate, etc). By combining the Profit By Click and Profit By Impression winners with the knowledge of why the other variables preformed best in specific metrics then one can leverage the entire testing statistics in putting together new ads, landing pages, and tests.
The above test was run with a static item cost ($25). For many sites, the total order and average order amount will also be statistics you wish to measure.
In the below chart, the base numbers are the same, the only number that was changed was Sales $$. This is the combined value of all the sales made during the test period. Of course, if the sales value is changed, the profit margin will also be affected. Average sale was added as a metric.
The above numbers can be analyzed just like the first chart above. It's important to show how average sale and total sales can affect the test. The two previous winners are no longer the top performing tests. Test D, which only won on CTR earlier, is the clear Profit by Impression winner. Test E is the clear Profit by Click Winner.
In this scenario, besides the information that we already walked through that needs to be analyzed, there is an emerging trend. Test E is full of converting traffic (which could be the keywords, ad copy that speaks only to buyers, etc). Determining why it has such a great conversion rate and then applying those results to Test D (which has the higher CTR, 3rd lowest CPC, 2nd highest conversions, 2nd highest sales, 2nd highest avg sale, highest profit) would bring about some pretty significant changes and improvements.
These tests are but two simple scenarios in a sea of online metrics. One could easily add page views per visitor, time spent on site per keyword, Value of a Lifetime Customer, catalog requests, etc. Metrics are easy to add, what's important is assigning a dollar amount to those metrics. Determine what a phone call is worth, what a catalog request brings in sales, what an email does for your bottom line. Determine not only what action you want someone to take - but what that value is to your company.
Profit by impression (PBI) and Profit by Click (PBC) are the best metrics to determine when split testing ad copy or landing pages. One can determine profit by impression/click numbers by individual line of an ad copy, by the entire ad copy, by landing page, etc.
When split testing, think about the actual goal: user action. By understanding how each line of an ad interacts with a landing page and with a search query, by understanding how a landing page is affected by a keyword, by understanding how a keyword interacts with a specific ad, by understanding all of your metrics you can finally determine which is the proper combination to be running.
If you know these statistics:
You can derive these statistics:
These are elements in a PPC campaign that you control. If you can measure it and understand the metric, you can change it. Knowledge of metrics is key. Knowledge of what to do with the metrics is crucial.
Profit by Impression & Profit by Click are still underutilized actions in the world of PPC; however, they are concepts that have been successfully implemented for several years.
Only by understanding how the entire PPC cycle works, from query to ad copy to click to landing page to user action can one really understand the value of advertising.
Understand the value. Measure the value. Improve the value.
Three simple steps to successful advertising.
Split testing Ad Copy & Landing Pages: [webmasterworld.com...]
A/B Testing (supporters) [webmasterworld.com...]