|"Google Experiments" vs. "Google Website Optimizer"|
I must say, as a long time fan/user of Google Website Optimizer (GWO), I'm extremely disappointed Google is gettind rid of it and replacing it with Google Experiments.
Since we are being forced on the transition, I did a test experiment with Google Experiments. The biggest drawback I found was the added tracking that they attach to the end of a URL. So for example, if I set up an experiment on http://example.com/index.html and http://example.com/index-b.html , when I visit example.com in my browser it could appear as...
This change in URL is enough to throw off some visitors and adds an extra variable, making it impossible to get true split testing.
With Google Website Optimizer, you were able to setup a multivariate experiment that wouldn't change the URL. I could set up an experiment for http://example.com/index.html, put the code from http://example.com/index-b.html in GWO, then, every time I visited my site it would come up as http://example.com/index.html, showing both versions of the page under that one URL.
The next obvious complaint is not having Multivariate experiments. This was invaluable when trying to test multiple things on a page at the same time. With Google Experiments, this lost feature limits our options of what we can do in testing.
Finally, it says in my Google Analytics account that experiments are limited to 3 months. I've had many experiments that take longer than 3 months for Google to declare a winner. While most of my experiments do finish in under 3 months, this is just another thing I have to worry about if I want to do testing with Google Experiments.
What do you think, do you share or oppose my complaints?
[edited by: tedster at 9:42 pm (utc) on Jul 3, 2012]
[edit reason] switch to example.com [/edit]
No doubt in my mind, if you are concerned about the url change affecting your test results, you are ready for something more intensive to do multivariate testing.
Here's one thing I would do - first use (or program) analytics that measures browser data - the kind where you can see how people scroll around your page, how long they stop in any given section and so on.
Thanks for the reply, Tedster. Yes, we use Clicktale and other software... "the kind where you can see how people scroll around your page, how long they stop in any given section and so on." I totally agree, Clicktale can indeed give ideas on how to improve the page, which you then split-test to large numbers.
But with ab/split testing, we might test anything, from an idea that's as simple as improving an image, or something as major as a totally new presentation of a page. These types of changes can't be handled by Clicktale-type of software.
And re the "url change affecting your test results", my boss is rather anal about eliminating ANY variable that can distract. While it doesn't make a difference when you "luck" into a 20% conversion rate improvement, we usually test to high numbers in order to be sure of a 2% increase in sales. If only 1/50 people notice the URL difference, we're not sure what that distraction means.
One final note, I'm kind of surprised there aren't more people who are actually bothered by this. The loss of multivariate is a big loss, the URL change is a step backward in sophistication and anything else important that I've noted.
Does anyone actually use GWO much and if so, have you compared GWO to the new Google Experiments?
|...is a step backward in sophistication |
This seems to be a common factor in all of the recent Google product "upgrades" from WMT to Analytics and others.