Tracking Keyword Ranking Position with Google Analytics

[travelin cat]

Google announced a change in the format of the referring URL as mentioned in this thread : [webmasterworld.com...]

Recently there have been a few blog posts summarizing the steps that allow you to set up a series of filters that when properly deployed will provide you with ranking results displayed next to the referring keyword string like this:

keyword+keyword (Rank: 1)
keyword+keyword +keyword (Rank: 1)
keyword+keyword +keyword (Rank: 4)

Here are the steps that I have sucessfully used on one of my sites:

1. Create a New Profile

Create a new profile within Google Analytics so that you do not interfere with the data that you may already have set up.

To do this, sign in to your Google Analytics account and click on"Add Website Profile" on the bottom of the page. Then choose "Add a Profile for an existing domain", choose the domain and give the profile a unique name.

2. Creating the Filters

NOTE : It is important to create these filters in the following order

2a. Include Organic Serps

Filter Type: Custom filter -> choose Include
Filter Field: Campaign Medium
Filter Pattern: organic
Case Sensitive: No

2b. Include Google Serps

Filter Type: Custom filter -> choose Include
Filter Field: Campaign Source
Filter Pattern: google
Case Sensitive: No

2c. Restrict to New Google Referral URL

Filter Type: Custom filter -> choose Include
Filter Field: Referral
Filter Pattern: google.com/(search¦url).*\bcd=\d*
Case Sensitive: No

2d. Construct Ranking String

Filter Type: Custom filter -> choose Advanced
Field A -> Extract A -> Choose Referral -> \bq=([^&]*)
Field A -> Extract B -> Choose Referral -> \bcd=(\d*)
Output To -> Constructor -> Choose User Defined -> $A1 (Rank: $B1)
Field A Required: Yes
Field B Required: Yes
Override Output Field: Yes
Case Sensitive: No

3. Viewing the results

Once you have created the new profile and filters, it may take a couple of hours to start seeing results. You will find the new keyword/ranking data in any of your reports when you change the dimension to User Defined Value.

[edited by: engine at 4:16 pm (utc) on May 13, 2009]

[MadeWillis]

excellent, thank you!

[travelin cat]

I should point out that this method will only show results for queries using the new format as mentioned above.

We have noticed that maybe 8-10% of all Google searches on our site use the new format.

[netmeg]

Field A -> Extract B -> Choose Referral -> \bcd=(\d*)

Isn't that supposed to be Field B?

[travelin cat]

Netmeg,

You're correct.

Field A -> Extract B -> Choose Referral -> \bcd=(\d*)

should be

Field B -> Extract B -> Choose Referral -> \bcd=(\d*)

Thanks for the catch

[netmeg]

All I'm getting is (not set) so far.

[JeremyL]

This won't work exactly as shown. Every single blog post I have seen on the subject is wrong because they are only taking into account the normal search results. universal search results will make that data collected look all haywire because the cd= parameter can be used to show ranking within a set of sitelinks or in universal local, ect results.

Each non-normal SERP ranking with the cd= parameter also has an additional parameter marking it as part of universal search. You will need to take that into account to get the data right.

Do a search that products site links for a site and compare every sitelink and all their parameters to the normal SERP parameters and you will see what I mean.

[edited by: JeremyL at 3:55 am (utc) on May 14, 2009]

[travelin cat]

Actually it works very well, we have been using it for over a week now with the stated results.

Netmeg, to see results, go to:

Traffic Sources -> All Traffic Sources

You should see results for google/organic

Click on it and look for the "Dimension" pop up, it will default to None. Choose "User Defined Value"

[JeremyL]

@travelin cat

It works, yes, but is potentially inaccurate. Anyone who clicks on your link from any universal search listing (news, product, local, ect) or site link will have a cd= value that is relative it's position within the site links or universal search block, not it's position in the overall SERPS.

There are additional parameters that are added to indicate if a listing is from within site links or universal search.

So for your filter to be accurate, you would need to take into account those instances by adding one or more rules to your setup depending on if you just want to filter out those oddities or actually find some way to count them.

[netmeg]

Netmeg, to see results, go to:

Traffic Sources -> All Traffic Sources

You should see results for google/organic

Click on it and look for the "Dimension" pop up, it will default to None. Choose "User Defined Value"

Nope, gets (not set) Copied and pasted directly from this post.

[freejung]

I'm having trouble with this too -- I've never created a second profile before, so I may be doing something wrong.

I created a second profile for my site, but it appears not to be collecting data. The Status says "tracking unknown" and there is no data in any reports for the new profile. Though the tracking code is profile-specific, Google's documentation says you don't have to install new tracking code if you make a duplicate profile of an existing domain.

Is there some step I'm missing?

[travelin cat]

You do not need to install additional code, just make sure you do this to make a new profile:

...sign in to your Google Analytics account and click on"Add Website Profile" on the bottom of the page. Then choose "Add a Profile for an existing domain", choose the domain and give the profile a unique name.

This will allow GA to use data already being collected for an existing profile and apply it to the new profile.

[freejung]

Yep, did that. With two different sites, actually. One still uses the Urchin tracking code, so I thought that might be the problem, but the other site uses the new code and has the same problem.

It's been more than 24 hours, but maybe my data just hasn't posted yet. I'll post again in a couple of days if it's still not collecting data.

[chewy]

Any chance that the pipe (¦) should actually be a solid pipe --- or vice - versa?

I learned this the hard way over on the Apache forum that WebmasterWorld somehow handles this oh-so-special character differently than logic would have it.

[travelin cat]

chewy, I think you may be on to something. It should be a solid pipe: shift backslash.

[chewy]

12 hours into this and it isn't reporting anything yet.

New profiles usually show general traffic as well as the filtered results - this new profile isn't even showing that yet.

Will continue to monitor and report.

[Chris_R]

Tried the first one and it didn't work for me (I am pretty familiar with Analytics - but regular expressions aren't my strong suit)

So I:
1) Tried chewy's suggestion of the ¦ vs ¦ and that at least seems to pass one of the online Google Analytics Filter testers.
2) Changed the profile to use ¦ instead of ¦
3) Made another profile not including the fourth step - so I can see what is going on if this doesn't work.

Looking forward to seeing this data and will report back if it works for me.

[Chris_R]

Ahhhh - WebmasterWorld Forums turns solid pipes into ¦. Now I get it....

[travelin cat]

Sorry about that. I didn't know that at the time of my posting or I would have pointed it out.

[Chris_R]

Now it works fine - thanks!

[robzilla]

Interestingly, all hits so far in this profile are showing a 100% bounce rate. Am I correct in assuming that these filters limit this particular profile to being useful only for showing ranking data?

[travelin cat]

I'm seeing normal bounce rates on my data, seems about the same as my original profile.

[chewy]

Google Chris Abernathy for more.

[freejung]

I fixed my pipe character (should have thought of that!), and now it's working fine.

Wow, the rank varies quite a bit even for keywords where I've had stable ranking for years. Interesting.

Also, my bounce rates are normal, so I'm not sure what's going on with robzilla's results.

[travelin cat]

the rank varies quite a bit even for keywords where I've had stable ranking for years.

I noticed the same thing. One KW phrase has been #1 for years (I thought) and now I see that it is #2 occasionally.

[chewy]

OK, so I am finally seeing data 3 days into this.

I am seeing numbers like 2 out of 28 Google searches per day are showing up in this profile.

This isn't good enough to show the client - anybody else seeing under 10% of the data represented?

[robzilla]

I thought I'd give it a couple of days, and I even tried removing and then readding the filter, but to no avail. Only organic Google traffic is reported, with uncommonly high bounce rates; all other referrals are simply not included. I'm using the GA tracking code from about half a year ago, and noticed it has been changed since. Could that have something to do with it? I triple-checked the filter settings and they look all right.

anybody else seeing under 10% of the data represented

Yes, the organic Google traffic reported in this profile, with the filters in place, is roughly equivalent to 10% of the total amount of Google traffic reported in the regular profile, without the filters.

[chewy]

Looks like the OP travelingcat predicted that only 8 - 10% of queries are using the new format - so that would be consistent with what Robzilla and I are seeing.

If I understand this, this is due to this new parameter (cd=)that G is planting in the string, right?

Can anyone please speculate, might this be because only 10% of traffic actually qualifies to earn this new parameter, or that Google is only running this on 10% of queries as they are in a testing pattern and may roll this out further?

Per what I am reading on chrisabernethy (not Chris Abernathy - my mistake!) I will try removing filter # 3 and see if this makes any useful difference.

I think I need to study some logfiles!

[robzilla]

The new referrer URLs will initially only occur in a small percentage of searches. You should expect to see old and new forms of the URLs as this change gradually rolls out.

[analytics.blogspot.com...]

If I access www.google.com directly, which I rarely do, I get the new URLs. I do not, however, when I use the search box in Firefox or when I search from my iGoogle page. International Google homepages also seem unaffected.

I'll join you in removing filter #3, and may also get rid of #1 and #2.

[robzilla]

Include Pattern: This type of filter includes log file lines (hits) that match the Filter Pattern. All non-matching hits will be ignored and any data in non-matching hits is unavailable to the Urchin reports.

[google.com...]

Doesn't that mean that using the include patterns as outlined in the tutorial automatically excludes all other traffic?