Forum Moderators: DixonJones
Changing tools can be a frustrating and difficult process. Numbers and intelligence form the bedrock of business decision making, and over time you grow to trust your numbers to the point that when you get conflicting data (as always happens when you change tools) it can throw you into a panic and cast doubt on everything you have done.
Having been through this process several times, I thought I would share some guidelines on how to transition from one tool to another.
1. Don't Panic. The first time you compare numbers, they will be off.
2. Do not try to get your old and new packages to match up exactly. They will not. Instead strive to get them within 3-5% of each other.
3. Understand the basics of both packages before you switch from one to the other. If for instance you are using a log analysis tool, transitioning to a javascript based one is going to be a more difficult process because they are two fundamentally different approaches to recording traffic.
For instance, a javascript solution will not record traffic if a visitor has javascript turned off while a log tool will.
4. Understand your website structure very well before switching. If part of your site is hosted outside your servers, then a javascript solution can still track it where a logger may not. Try to identify trouble parts of your site before switching.
4A. Some solutions have flexible filters that you can build and run traffic for at any point in time. Others need to define site taxonomy in the tag and cannot go back and run numbers for periods that are not defined.
Know what kind of solution you are moving to, if it requires you tag before you record then make sure you get your filters setup correctly the first time out.
5. Definitions and configurations are key to changing packages. No two stats packages define things the same and no two packages will record traffic the same. Here are some things to look for:
A. Cookies: some packages can read your cookies, others will issue their own and use them. If you rely on data in your cookie for tracking, make sure the new system can read it. Also make sure that a persistent cookie is defined the same way. You don't want one system only tracking uniques for 30 days while another does it for 10 years.
B. What is a page? If your site is more than just straight HTML then you need to make sure that what you define as a page is the same in both packages. Sometimes Java is just code running within a page, sometimes the page is served as a Java extension. If you have weird extensions like .action or .do or .whatever, make sure that you define them as pages when appropriate. Likewise make sure that the package is not calling something a page that you do not consider a page.
C. What is a visit? Some packages will count you as a visit if you only request a file and not an actual page. This can wreak havoc if your content gets hotlinked a lot or if you have viral widgets like quizzes that people can put on other sites.
D. What should get excluded? Some packages are smart enough to exclude known bots and crawlers automatically. Others require that you manually set this up.
Make sure your exclude filters in both packages are identical. Exclude internal traffic and any monitoring URLs or services.
6. Troubleshooting and Accuracy: Once you have your new system configured and spitting out numbers, now you need to figure out if they are correct. Here is a process to go by:
A. Make sure you are getting all of your traffic first.
If you have multiple servers or subdomains, make sure that each one is recording and that the traffic on each looks proportionally correct. If you do not see traffic for a subdomain then go back to the configuration and make sure your config is setup right, then make sure your tracking code is on that subdomain.
Run a traffic by URL report. Make sure that traffic is recording for your major URLs. If some are missing then check the page to see if your tracking code is on it, and check to make sure you are not accidentally excluding it.
B. Make sure you are not recording stuff that you don't care about. See if any file extensions you do not consider a page show up in your URL report.
C. Run a closed controlled test. Create a directory on your site that no one knows about, then personally generate a known amount of traffic on it, and see if both systems match up.
D. Look out for dupes. A few good numbers to look out for on this are:
Visits per visitor
Page views per visit
If either double or triple then you have a problem.
One thing that can trigger this is crossing subdomains within a session. Make sure you only count someone once throughout the entire site.
E. Start small and work your way out. Get both packages to be close for one page, then try one directory, then try one subdomain, then try the entire website.
The smaller your report, the easier it is to identify a problem.
I hope this will help you switch packages. A few final words of wisdom....
Don't believe your new vendor when they say it will be easy.
Prepare your management for a rough go of it.
The key to switching packages is details. Don't get overwhelmed, just make a list and work through it item by item.
