I guess the question here is how much data does Adwords need to determine an ads performance? Sofar I've seen that Adwords raises the percentages in which it shows new ads although (in some cases) they have significantly lower CTR's.
The question is when does the optimization (which you would expect to favor higher CTR ads) kick in? (after X impressions and/or Y clicks and/or Z days)
I guess this is a question for the propellerheads that AWA talks to from time to time :-)
I had to turn off optimizing this morning, though. One of the ads was showing around 90% of the time. That's not optimizing, that's crazy. And is ultimately self-defeating because you're serving stale ads to people over and over again. That's why you have different ads in an ad group, to avoid staleness.
The AdWords experience just gets better and better. Good God what are these people doing.
Essentially, the system learns as it goes, and is impression-based rather than time-based.
So the more impressions your ads have had, the more accurately the system will be able to judge which ones are emerging as the most successful.
The system is more accurate at 1000 impressions than it is at 100 impressions. And more accurate at 5000 impressions than at 1000. And, as it gets more accurate, it shifts the delivery more toward the most successful ads.
You are correct that CTR is the measure, BTW.
Hope that makes sense.
Now, I think I'll go to 'a well know online auction site' and see if I can find a vintage propeller beanie. All of a sudden, I want one!
AWA
I understand what your are saying but have a questions on how the optimizing process itself works. First adwords needs to have X impressions and Y CTR to determine how succesful and Ad variant is and to compare it with others.
What I noticed over the weekend though was strange. I created some new ads to test the optimizing, next to a single ad that already existed. After 2 days we had had thousands of impressions and 50 or so clicks per ad. At that point it was clear to me that the new ad had a significantly lower CTR than the old ad. Adwords however continued to increase the visibility of the newer, less performing, ad which does not sound like optimizing to me.
The only reason (next to a bug) for this behavior IMO is that Adwords needs a certain level of impressions/clicks to know whether the CTR of the new ad is statistically relevant/reliable, hence bringing up the visibility to get there sooner.
This behavior however makes me (and maybe others as well) nervous as it "seems" to optimize for the lower CTR ad (while it may just be tuning the system). So I guess my question is how much clicks/impressions are needed before Adowrds deems the CTR significant? or are we dealing with a bug here?
I'll post again when I know more.
By the way, I have to say that I like the new schedule I've arranged that allows me to vist WebmasterWorld more often during the day. Previously, I was only able to post in the late evening. I think this is much better.
AWA
By the way, I have to say that I like the new schedule I've arranged that allows me to vist WebmasterWorld more often during the day. Previously, I was only able to post in the late evening. I think this is much better.
You and us both :)
An Ice Cream has been requested for you, let me know when they give it to you, and also if they dont, let me know.
Shak
Once again the people in the next cube are wondering why I just busted out laughing.
You crack me up!
:) AWA
am I mis-informed about the office move?
Shak
I knew you were serious. That's why I cracked up!
So, on my desk right now: Ben & Jerry's Cherry Garcia. Was there when I came back from a meeting. Not even melted yet.
GG, come on over, I'll share!
Thanks Shak - GG is right: Too funny!
AWA
Robsp, I neglected to ask you one essential question.
Will you confirm for me that you are referring to multiple ads within one Ad Group?
Or are you referring to multiple ads (in multiple Ad Groups) within one campaign?
The reason I ask, is that the new feature is only designed to work with the first scenario, and not with the second.
In other words it monitors multiple ads within the same Ad Group, and starts to show the 'best' one more frequently.
AWA
Hope you enjoyed your icecream :-)
I can confirm that we are talking about 2 ads in one ad group yes and I like to know how the optimization process does its optimization.
As I'm in Europe I'll call it a day for now....
The optimzing is working now for some of my ads. My assumption that Adwords first needs to time to figure out how the new ad is doing seems correct. The open question is how does it determine this?
I believe it serves both ads an equal amount of times until one ad gets a better CTR. Then it serves that ad more often.
At what point (how many impressions) does it decide which ad is better?
That's the key question!
Rob
Whew! Busy morning - had to finish a few tasks before I could visit Webmaster World.
So, I've have checked with a tech person, and it turns out I was mistaken (in post #4 above) when I suggested that the system is not concerned with time.
I've been informed that the system 'scales' (i.e. learns over time) based on the number of impressions, but that it has no set impression limit after which it starts working. In other words, this is not a 'go' or 'no go' situation, where it starts after 'x' number of impressions.
Also the system updates daily - although the time at which it does so is not fixed.
All of this implies to me that the longer the system runs, the more accurate it is, and that one may not see 'instant' results the first day, as it may not have updated yet.
Hope this info helps.
TomWaits, you wrote:
...even though I have the optimizer unchecked, it's still optimizing. I'm getting 1 ad showing up 90% of the time. The new system is buggy.
I'll pass this info on to the tech folks. However, I wonder if this could be a case of the system evenly showing your ads, and adding the stats on top of the previous 'skewed' stats accrued when you were using the optimization feature?
AWA
I wonder if this could be a case of the system evenly showing your ads, and adding the stats on top of the previous 'skewed' stats accrued when you were using the optimization feature?
Unfortunately, I don't think that's it. I turned it off yesterday morning. Stats for today are still all screwball for some campaigns, while others are ok.
The ones that are ok have less variability than the ones that are still screwy. For example, for one of the ones that still off and have big variability, there's one ad that is about double the click-through rate of the other ads in that ad group, and that's the one that continues to show 90% of the time. The call to action means that the ROI isn't that great with this ad, but as I mentioned I'm trying to accomplish different things with different ads within an ad group, so I want them all to show evenly just as before.
It makes sense and is also consistent with what I see. You just need the nerve to sit through one or 2 days of serving increasing percentages of badly performing ads.
I'm getting confortable with the system as I see it in action and used it again today in a few of may new campaigns.
There must be certain criteria that Adwords is using but I guess that's either to confidential or to detailed :-)
Thanx!
I've been informed that the system 'scales' (i.e. learns over time) based on the number of impressions, but that it has no set impression limit after which it starts working. In other words, this is not a 'go' or 'no go' situation, where it starts after 'x' number of impressions.
How can the system determine which add is better if it doesnt give a minimum number of impressions before it starts favoring one add over another?
In other words, how can one add 'catch up' to another that has a better ctr if it isnt getting served as much?
Is it better to disable the auto-optimization for new adds until there is a reasonable number of impressions, or should you just let the auto-optimization handle it?
Thanks.
AWA
Is it better to disable the auto-optimization for new adds until there is a reasonable number of impressions, or should you just let the auto-optimization handle it?
An answer was never provided...and I now have the same question myself. AWA can you shed any light on this question.
Generic situation:
I have been running adwords for a few weeks with the optimization turned ON. I have one campaign with seven adgroups each containing approximately 3 ads. This evening I am adding a few new ads to a single adgroup.
What is the best approach to having these new ads receive their fair share of impressions? Keep the auto-optimization ON or turn it off so they rotate regularly for a few days...then turn it back ON.
Does turning the auto-optimization ON and OFF reset the "knowledge" the system has gained for particular ads in an adgroup? Or will it retain the previous knowledge of older ads when the switch is flipped back on and then recalculate the optimization based on "old knowledge" plus recently gathered knowledge?
Just trying to figure out the pros and cons of adding new Adverts and what happens with turning On and Off the optimization.
Thanks in advance for any input. Sorry if it sounds confusing.
Todd
I have one AdGroup with three ads.
Ad 1 has a 9.8% CTR and is shown 96% of the time.
Ad 2 has a 9.6% CTR and is shown 2.8% of the time.
Ad 3 has a 9.1% CTR and is shown 1.2% of the time.
This seems like a huge difference for ads that are fairly close in CTR.
In other groups, we see ads with 10% CTRs shown 55% of the time while an ad with a 5% CTR is shown 45% of the time.
I think this is still very much in testing.
Is it better to disable the auto-optimization for new adds until there is a reasonable number of impressions, or should you just let the auto-optimization handle it?
I've been trying to get a little more 'depthy' technical understanding of this feature. However, as things are especially busy here, with lots of advertising activity (!), I've had trouble connecting with the right folks. I'll keep at it though.
That said, I think the above question really boils down to advertiser preference. I guess if you're not certain, I'd lean towards suggesting that you disable the feature, and let the ads run evenly for a time.
Does turning the auto-optimization ON and OFF reset the "knowledge" the system has gained for particular ads in an adgroup? Or will it retain the previous knowledge of older ads when the switch is flipped back on and then recalculate the optimization based on "old knowledge" plus recently gathered knowledge?
toddski07, this is the part I was hoping to contact a techy-sort about. I'm reasonably sure it "retains the previous knowledge of older ads when the switch is flipped back on and then recalculate the optimization based on "old knowledge" plus recently gathered knowledge", as you put it. ;)
I usually like to be about 99.5% certain (at the minimum) when I post, but on this one I am not up to that level. So give me some time to confirm before you quote me.
AWA
I think it is important to know...cuz if I have an old ad or two performing well and then turn off optimization just to test a couple new ones...I wouldn't want to start from square one again when turning the optimization back on.
Thanks again.
Todd
AWA...hear any more news on the effects of turning the optimization on or off?
Thanks.
Todd
What with a high seasonal volume of activity here, combined with a few 'sources' taking Holiday time off, it's taking me longer than usual to get an answer.
Haven't forgotten though - and will post once I find out. ;)
Sorry for the delay.
AWA
Every once in a while it tests again how the other ads are doing, which sometimes means that another add gets all impressions. This is handy when you add keywords to an adgroup and I would suspect that adding keywords may "trigger" G to try another round of optimizations.
But hey, I'm just speculating here....
