experiment testing the effect of visitor interaction on rankings

[Shepherd]

There's been a lot of talk of the many signals google is or could be using to rank websites. One of those signals could be visitor interaction statistics. Of course it has not yet been nailed down that they are using this as a ranking factor but I for one would think this would be a good signal for them to use if they could get clear data (and I think they can).

One of our sites was hit by the original penguin and never recovered for it's main keyword, it is also an EMD. Prior to April, 2012 this site ranked number 1 for the main keyword for about 2 years. For the past year this site has ranked consistently in the 80 to 120 range for it's main keyword.

For this particular site/niche google has a lot of data. They have data that we gave them via analytic and they have data from another business in the same niche that they invested in.

What we did this weekend was lower the price of our product to a point where the conversion rate has just about doubled from 2.89% to 5.68% (sitewide, all visitors). The bounce rate has gone from 39% to 26% and average time on the site has gone from 203 seconds to 290 seconds.

Today is day 3 of the experiment and of course it is way to early to expect to see any results. However we've noticed some interesting ranking changes today compared to a ranking check from Friday before we started the lower price test:

1. widget (EMD): Friday: #97, today: #56
2. widget product: Friday: #127, today: #39
3. california widget: Friday: #12, today: #5
4. widgets: Friday: #145, today: #82

It goes on and on down our keyword list. It's almost crazy to think that we could be seeing any results this quick from this change but we have made no other changes to the site and we have done no promotion or link building since April, 2012.

We are going to keep this test going for at least the rest of the week to see what kind of info we can get.

[Robert Charlton]

Interesting test. It reminds me that a while back I suggested to an ecommerce site owner that his prices were too high for his market. I don't know whether he's acted on it.

It's worth noting that the trick in any experiment is to isolate the factors involved. Here, I'd keep in mind that Google is in the middle of a "softer" Panda update, so there might be some ranking change independent of your price change.

After you run through this test, you might (if you want to risk it) try raising your prices again and seeing what happens... and then repeating the price drop test again. I assume that a faster and more random A-B type testing would not work for observing the effects of prices on ranking.

There are also longer term effects of price that might exist but which you might not see right away. Cheaper, while not always better, should obviously help when customers are looking for best price. I've seen a reputation for good prices (coupled with good service) attract inbound links in price-conscious areas online.

I'm looking forward to future reports. Thanks.

[Shepherd]

After you run through this test, you might (if you want to risk it) try raising your prices again and seeing what happens

Yes, that's what I was thinking also, if we get some definitive movement in the serps (more than we've already seen) I'd love to get some solid proof that visitor interaction is being used for ranking.

[Shepherd]

data that we gave them via analytic

It should be noted that we stopped running google analytics over a year ago when google invested in our competitor. So in this case if user metrics are being used to influence ranking google would be getting the data from somewhere else (chrome? ISP?)

[Robert Charlton]

I don't think it's Analytics. We've had a lot of discussions about where Google might get the data... speculating about everything from ISP data to DoubleClick... and Google may correlate what they see elsewhere with Analytics averaged over a bunch of sites, but Analytics by itself wouldn't even provide Google with a consistent data set.

If searchers come back to Google, Google probably has recent google.com cookie data on the searcher... no matter what the browser. See this thread for some interesting thoughts...

Panda Metric : Google Usage of User Engagement Metrics
http://www.webmasterworld.com/google/4302140.htm [webmasterworld.com]

I think they're measuring things like engagement, stickiness, and not coming back to search again on Google... and lower price might correlate with all of these things.

[SEOPTI]

I think this is some sort of Bing "Panda" and Goog might also be using this approach:

"And to the engine, well, “useful” is more aligned with what the searcher went for, with intelligence around engagement added from past similar queries."

[bing.com...]

[muzza64]

Shepherd, we tried something similar over a year ago but only for selected products and there was a noticeable improvement in rankings for those product pages which has remained to this day.

We undercut everyone by a mile and no competitors responded (although many have price match guarantees) so we still remain the most competitive on these products.

Things to note:-
1) Rich snippets show our product price which, being much cheaper, may have improved our CTR from Google for these products (which I believe is a powerful ranking factor).
2) Bounce rate improved for these products. I think this combined with a great CTR from Google is a potent combination.
3) Being ultra competitive for these products did generate more links, buzz, etc. than for other products.

Our conclusion was there were several factors potentially influencing the outcome which were all a consequence of the price reductions. My gut feeling is the improved CTR and bounce rate were the main factors responsible for the initial ranking improvements (within a week or two) and then links provided a longer term boost.

Unfortunately we couldn't afford to do it across the board so the effect was limited to these product pages. If we could be this competitive on everything I think the whole site would have seen a huge improvement in rankings.

Ultimately though what we did was make ourselves a little bit more unique.

[JS_Harris]

Congrats on the improving results but, in my opinion, it's highly unlikely that changes you made Friday have improved your rankings that much by Monday even with the improved visitor metrics. There is also a Panda rollout currently going on to muddle the test further and if that's not enough Google holds a patent to dampen rankings movement over time.

While your changes probably didn't hurt it's unlikely they could have helped so quickly.

[Shepherd]

I agree with you JS, we're going to try and un-muddy it by fluctuating our user metric stats and watching the rankings.

That patent you mention, if I remember correctly, was an attempt to keep webmasters from gleaning any real ranking information from changes they made to a website. We are not making changes to the website, we are changing the signals (user metrics) that google would be measuring (if they are) to determine ranking.

We're in kind of a unique position to be able to change the metrics pretty much whenever we want without making any real changes to the content or structure of the site. If google is using user metrics to determine ranking we will be able to see it.

[SEOPanda]

I think it's a coincidence.

If this was in fact the case, all you'd need to do is to have a few products sell for cost (or even way under cost), and make up for the lost profit with your whole site ranking higher and you selling other products for full price.

[Shepherd]

If this was in fact the case

You have to look at the big picture, we're trying to find out if user metrics are being used to determine rankings. We're not tying this to product pricing other than that it is what we are using to modify our user metrics (time on site, bounce rate, average page visits, ecom conversion rates).

whole site ranking higher

If user metrics are being used for ranking why would the whole site rank better? Why would product pages that don't have good user metrics rank well?

[Shepherd]

ranking update, day 5 of higher/better user metrics:

1. widget (EMD): Friday: #97, monday: #56, wednesday: #54
2. widget product: Friday: #127, monday: #39, wednesday: #31
3. california widget: Friday: #12, monday: #5, wednesday: #7
4. widgets: Friday: #145, monday: #82, wednesday: #85

Not much movement from the initial bounce. Will likely go back to the previous weeks user metrics this friday, hopefully we'll see the rankings revert at the same time.

[SEOPanda]

"If user metrics are being used for ranking why would the whole site rank better? Why would product pages that don't have good user metrics rank well? "

Your #1 example indicates the site itself is ranking better even though the product is the one with the lower price and lower bounce.

[Shepherd]

I see, it's difficult to be more descriptive without trampling the TOS.

This site sells 1 "product", a widget, and the domain is an EMD to the product's general name, widget. The examples, 1-4, are keywords the site is ranking for.

[frankleeceo]

One question for this test. What is the amount of traffic funneled in this test?

If traffic is too little, it may not make any statistical difference.

[Shepherd]

What is the amount of traffic

Good point, google search traffic is right at 2,700 unique/day. Total traffic is about 3,200 unique/day

[diberry]

This may be a completely stupid thought, but can Googlebot read the new prices? If so, how can we be sure they're not simply reacting to that because they assume a searcher would enjoy the user experience you're offering with that bargain? User metrics wouldn't even have to come into it.

[frankleeceo]

Thanks for the traffic answer. So with that number you pretty much gotten hundreds more sales. Did the actual traffic increase too with the ranking increase?

I am speculating that user metrics can go deeper within a site. A completed transaction may mean that user is completely satisfied to a point of completing the transaction. If google can read it they would place more importance on it rather than dwelling time, bounce rate, or any other metrics no?

Was it more profitable for the whole site after slashing the prices? With the ranking and traffic increase, what did it do to your bottom line? (I would reckon to keep the test going for longer period of time if you guys are making more money! Why not?). I do think 1 weekend of testing is not enough to get a truly great picture.

Maybe google cannot exact tell prices but rather by the increased transaction done by user...to determine that the site or product is of higher value?

I think I read somewhere Google can track downstream or visit flow. How many pageviews downstream within the site can Google track? Is there any way to find out? Maybe Google tracks users all the way to "transaction completed page"?

[Shepherd]

@diberry:

but can Googlebot read the new prices?

We don't use any markup or metadata but yes, the gbot can see the price. google has the data in this niche to compare prices, not sure if they would bother but I'm certainly not taking any theories off the table.

@frankleeceo: traffic is up 6% for Saturday through Wednesday over the same time period last week. Most of the traffic increase is coming from long tail searches that are ranking better. Obviously the examples I gave are not going to be seeing much traffic improvement, there's not a lot of difference in the traffic you get from being #127 and #31.

A completed transaction... If google can read it they would place more importance on it rather than dwelling time, bounce rate, or any other metrics no?

I would agree with that, especially if the site has been placed in the ecom bucket by google. Conversions are much more indicative of a good ecom site than the other metrics, an informational site would have to rely more on the other metrics.

Was it more profitable for the whole site after slashing the prices?

Revenue and profit lower but still profitable.

How many pageviews downstream within the site can Google track?

With Chrome I would have to think that the number of pageviews downstream google can track is limitless. 20% of our unique visitors use Chrome, 51% IE. I had no idea Chrome's percentage was that high until I just looked!

[Dymero]

Shepherd,

I'm happy to see this post, as it's another example of this phenomenon of almost instantaneous improvement in organic following a rise in sales overall, as I posted a couple weeks ago in another thread.

In our case, prices didn't change, and there was little change to our site, but people started buying in higher numbers throughout the site (not just organic) following the Independence Day weekend. The couple month period previous to that is typically slower for us.

So after sales started going up across the site, so did rankings. The opposite happened at the start of Q2, where sales across the site went down and so did rankings.

I don't think it has anything to do with sales specifically, but how it tracks visits. Google does indeed seem to be using some kind of user metrics to determine user satisfaction with a site, such as not returning to search results and more people suddenly visiting a site, and they seem to respond quickly to such rises or drops of qualified visits.

[Shepherd]

Week 1 complete. Prices raised to normal pricing today. User metrics returned pretty much back to expected levels with the exception of conversion rate which stayed surprisingly higher.

Just thinking out loud here, I wonder if the higher conversion rate/improved metrics over the last week may have tipped our site profile from informational to eCommerce.

Have not seen any ranking changes of any significance as of yet (and certainly don't expect to this soon).

Thanks to Rustybrick for mentioning this thread in his weekly search buzz recap. (hey Barry, you can always sticky me if you have questions)

As of right now, unless we see ranking movement back to previous ranks I'm going to have to say that there's not much compelling evidence of user metrics being the cause of the ranking increases we experienced.

Looking forward to seeing what next week brings.

[Shepherd]

Update: Have not seen any significant movement in the serps since deflating our user metrics...

[JS_Harris]

User metrics are not the only ranking factor and likely nowhere near as powerful as backlinks, I have a well ranked single page information site as proof. It has a 100% bounce rate as you'd expect from a one page site. For that reason the testing you are doing is hard to derive actionable data from but still, thanks for sharing!

Just thinking out loud here, I wonder if the higher conversion rate/improved metrics over the last week may have tipped our site profile from informational to eCommerce.

Probably not, if there is a product on the page it's virtually guaranteed to be treated as eCommerce and have to fight sites like Amazon. I have seen many informational pages about products get treated as eCom pages and fall 50 places in rank due to the sheer competition involved for the product. About a product = ecom even if its information about the product. About something you can do with the product = informational. Google's been polishing their detection on each.

[aristotle]

About a product = ecom even if its information about the product. About something you can do with the product = informational. Google's been polishing their detection on each.

They must not have polished it enough if pages only containing information about the product are consistently mistaken for ecom

[trinorthlighting]

We are experimenting on our ecom site with site speed. We have recently reduced the load time for our webpages, thus decreasing the bounce rate, increasing the page views and converting more. We are monitoring our keywords as well to see if the metrics give us a boost.

[Planet13]

@ Shepherd

Update: Have not seen any significant movement in the serps since deflating our user metrics...

Have you seen any moment since your July 31 update?

[EditorialGuy]

About a product = ecom even if its information about the product.

That also seems to be true of non-physical "products" that might fit under the heading of "services" or "experiences."

[Shepherd]

Have you seen any moment since your July 31 update?

We saw no significant movement based on the user metrics during our test. The test period however was very short, we might try a longer test in the future.

[Shai]

Love these tests. Thanks for that Shepherd. We should have an invite only test forum and share privately some other tests results. The problem with publicising something specific that you noticed, either during a test, or just during day to day running, is that if it's very good, it gets around pretty quick and before you know it, it's on a well known spammy forum.

[Planet13]

Hi again, Shepherd:

So what you are saying is that RIGHT NOW your ranking is still as it was on Day 5 of your test (meaning it HASN'T slipped back down) See the BOLDED rankings:

1. widget (EMD): Friday: #97, monday: #56, wednesday: #54
2. widget product: Friday: #127, monday: #39, wednesday: #31
3. california widget: Friday: #12, monday: #5, wednesday: #7
4. widgets: Friday: #145, monday: #82, wednesday: #85

Thanks in advance for confirming this.