Welcome to WebmasterWorld Guest from 107.23.176.162

Forum Moderators: mack

Message Too Old, No Replies

How "Direct Traffic" May Be Undermining Your Marketing

     
4:33 pm on Oct 4, 2017 (gmt 0)

Moderator

WebmasterWorld Administrator ergophobe is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 25, 2002
posts:8627
votes: 274


Wait, traffic is hurting me? I want less traffic?

Really, I don't. I'll take as much legitimatedirect traffic as I can get. But, a lot of traffic can show up as direct because of problems with the way your tracking and analytics is set up.

I'll discuss this in the context of Google Analytics (aka GA) here, but the same applies to any system you're using to track your marketing efforts.

What is Direct Traffic?

True, legitimate direct traffic is traffic that comes to you through people typing your URL into the browser, using a browser bookmark or other "direct" methods like that.

This is as opposed to coming through another source, such as clicking a link on another non-search website (a referral), doing a search in Google or Bing and clicking one of those links (put in the Organic channel by GA).

By default Google segments your data into a set of "channels" and gives you the definitions of those channels here: [support.google.com...] You can modify those default channels, but we'll get to that later.

Are you wasting your marketing efforts?

So Direct traffic is good, right? People are typing in your URL. They are loyal fans, right?

That might be true. If you just took out a Superbowl ad with your URL on it, that might get you a ton of direct traffic too. But for most of us, direct traffic will be a fairly modest contribution to the overall mix. If it's a large contributor, then it might mean that other sources of traffic have tracking problems and so you can't tell whether all that money you're spending on ads or effort you're putting into social media or your newsletter is working.

Ideally, you want only direct traffic to show as direct traffic and you want to track your marketing efforts to know which efforts are yielding results.

What Causes Over-Reporting of Direct Traffic?

Lots of things. Any time your tracking breaks down, it's going to drop those sessions into direct.

Cross-domain problems

If your site has shopping on one domain and checkout on another, you have to make sure that your analytics can recognize that when someone goes from shopping to checking out, it's the same person and the same session. This is called cross-domain tracking.

This is a common setup. I work with hotels who all have their informational website, but when it comes to making reservations, you go to the Sabre system. Until recently, all Shopify shops worked this way. There are many examples.

If you see a very high number of conversions where the landing page is your checkout page, you might have a problem with cross-domain tracking. That's a whole topic in itself, but if your business spans multiple domains, especially your checkout process, you'll want to research cross-domain tracking.

Email

If you send an email newsletter out and someone reads it from an application - Outlook on the desktop, their Gmail app on their phone, whatever - that will be direct traffic. They click on a link in the email, but that email has no referring website. So it's similar to a browser bookmark and will get counted as direct traffic

How do you get around that? You want to make sure you include UTM tags for source and medium, and that the medium is "email." You can also tag for campaign. So your URL would look like this

https://www.example.com/offerpage?utm_source=newsletter&utm_medium=email&utm_campaign=spring2018

If that's confusing, don't worry. You don't have to understand the details because Google provides a tool for building URLs for your campaigns
[ga-dev-tools.appspot.com...]

Just remember: if you are sending a newsletter out, every link to your site should be tagged with UTM parameters. You don't have to get fancy. You can use the same parameters on every link. As long as you have the source and medium, you'll be able to track that conversion back to your newsletter.

Secure to Non-Secure Referrals

Still running http? Look, it's time to buy a consonant! Get that S on the end for one reason: all referrals from https sites to your site are now going to show up in the direct bucket. As everyone else transitions to https, you are losing insight every day to which partners are most valuable.

I know some people still have their worries about https. Get over it. Just the better data is worth it. I've never seen traffic or rankings fall on a properly rolled-out https conversion.

Tracking Blockers

More and more people are running Privacy Badger. Sorry, but you can't do anything about this.

I've done a study on a decent size website and measured tracking blockers over a reasonable number of pageviews (I think it was about 8,000, which was two days of data for the site in question) and found that 7% of users were actually blocking Google Analytics.

How did I do this? I served one image if the ga() function was available and another if it wasn't. Since GA uses first-party cookies, if the function is available, tracking should be working. Subsequently, I found an article by someone who did a larger study and got a similar number.

If you have a tech-oriented site, that number could go up. But except on, say, a forum on web security, I wouldn't expect it to go to 30% or anything like that.

Off-line Ads

This is a case where people actually type in your URL, so it really is Direct, right? Sure, but no. Ideally, you would like to track that ad.

The simplest way to do that is to give a URL with a short "vanity" URL on it that gets redirected with all the UTM parameters you need. In other words, let's say I'm offering a spring special for radio listeners on the John Doe show.

The announcer will give the URL as example.com/john

But you will redirect that to
https://example.com/spring-offer?utm_source=johndoe&utm_medium=radio&utm_campaign=spring2018

That's just one example. Radio is, of course, not a default channel grouping in Google Analytics, so you would want to set that up, but if you're spending that much on radio, it's worth the 15 minutes it would take to set that up

Summary

So in short, direct traffic is great if what you're looking at is thousands of true fans who just type your URL in. Congratulations. You don't need Google or Facebook or advertising. You've crushed it.

But that's not likely what's actually happening. In all likelihood you're seeing
- broken cross-domain traffic
- improperly tagged email campaigns
- improperly redirected and tagged offline campaigns
- secure to non-secure referrals
- tracking blockers

You can't fix all of those, but if you want to figure out where your efforts are paying off, you want to fix as many of them as possible.

What About You?

Have you had issues with over-reported direct traffic? What was the source? What did you do about it?
10:48 pm on Oct 4, 2017 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12913
votes: 891


Great topic ergophobe

a lot of traffic can show up as direct because of problems with the way your tracking and analytics is set up.
I eventually stopped using all 3rd party visitor stats reports, even GSC & GA. IMO they all fail in some areas, Direct Traffic* being one of the major issues.

Referrer data has been compromised so much for the last several years: Incognito settings, Do Not Track browser extensions, HTTPS to HTTP sites, and others have all contributed to the ambiguity of where the site visitor came from.

I rely on my raw server logs for most all info and have just learned to grep what data I want from there.

*I think real Direct Traffic is an extremely valuable metric. It reflects how established your branding is. I would love to get a more exact stat.
12:18 am on Oct 5, 2017 (gmt 0)

Moderator

WebmasterWorld Administrator ergophobe is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 25, 2002
posts:8627
votes: 274


>> real Direct Traffic is an extremely valuable metric

Not just a valuable metric, but valuable traffic. It's your fans who are coming for free.

>>I would love to get a more exact stat.
And there's the rub!

Your tracking solution depends on your situation. If your reservations or checkout is on a separate domain and you're using raw logs, you still have the same problem with cross-domain tracking and you'll have to confect a solution on your own - passing a session hash as a GET or POST parameter, for example.

And of course, everything above applies to your email newsletter whether tracking via raw logs or a third-party solution, whether client side or server side (and GA can be either or both, by the way, but that's another topic). That is, the UTM parameters would work just fine, but since you're confecting your own tracking solution, you can use any parameter you want are not tied to use the utm_* parameters.

How you skin this cat will vary by your tracking methods, but the key thing is to shave off as much "pseudo-Direct" traffic as possible from your stats.

All of the above assumes
- you have a variety of active marketing efforts that are burning money and labor and you want to figure out which ones are worth it
- you are able to track through to conversions that can be boiled down to a dollar value to set against ad and labor costs

So the above might not help much for a site where the model is to publish content and sell ads. I don't have much experience there.

You should be able to apply similar methods to an affiliate site though. You would just have to track all outbound clicks, be able to assign a value to those clicks and, having done that, track the traffic back to its source.
12:58 am on Oct 5, 2017 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12913
votes: 891


real Direct Traffic is an extremely valuable metric

Not just a valuable metric, but valuable traffic. It's your fans who are coming for free
"metric" is not just a number, by metric I am referring to the traffic and all that it means.

Direct traffic means that my visitors know who/what my site is, have likely visited before and are returning because they find value. They are likely to share this with others through word of mouth, social media and placing incoming links. These are the people I like. This is the metric I want to fully utilize.

Building and expanding direct traffic should be the mission of every website. What I do is to keep track of the search terms/phrases visitors used when they came to my site the first time (and possibly returned.) By offering that context they may return, having found what they were looking for. I see what pages they visited and what other pages they went to. That part of the data I grep from my raw logs.

I play an active role in how, where & why my site is shared via social media. I have accounts at most resources and have created context specifically for SM use. I engage other users and build branding. This also expands direct visits.
12:15 pm on Oct 6, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Nov 15, 2001
posts: 1732
votes: 36


Good post ergophobe, I do have an issue with traffic getting dumped into GA direct when it doesn't belong there, just getting a handle on it now. As keyplyr mentioned genuine direct traffic is great, and to be encouraged. The top 10 websites I often visit are all typeins for me, and I don't hesitate to recommend them to others!
3:14 pm on Oct 6, 2017 (gmt 0)

Preferred Member

5+ Year Member

joined:Jan 6, 2011
posts:487
votes: 12


Eh, aren't all search visits on Safari reported as "direct" visits as well as a lot of search visits from the address bar in other browsers?
7:35 pm on Oct 6, 2017 (gmt 0)

Moderator

WebmasterWorld Administrator ergophobe is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 25, 2002
posts:8627
votes: 274


>>aren't all search visits on Safari reported as "direct" visits

Isn't that just mobile? In mobile, Siri autocomplete in Safari will suggest an actual link, so that will come through as direct.

There was also the issue back in 2012-13 where iOS Safari users (not Safari per se, but specifically on iOS) doing Google searches showed as direct, but that was corrected. If that's happening again, can you provide a link? I don't know about that issue.

RE the 2012-13 issue:
[searchengineland.com...]
[searchengineland.com...]

But in any case, your broader point is certainly true: there are many sources of "fake" direct traffic.

By the way, you can test such things by filtering the Real Time results in GA so that only you show (filter by city and UA for example; if, like me, you are in a very small community, that's usually enough to make me unique on any site I deal with), and then follow yourself around. You can see when sessions get dropped and so forth.
8:05 pm on Oct 6, 2017 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12913
votes: 891


aren't all search visits on Safari reported as "direct" visits

Isn't that just mobile?

Just a note: Most all new *touchscreen* computers support apps that were formerly only supported on mobile devices (phones & tablets.) The same Bing or Google Search app is used on a Chromebook, and AFAIK iPhone search apps can be used on newer Macbooks. Window phone apps are supported on Windows10 computers.

The distinctions between devices continue to blur.
[webmasterworld.com...]
10:01 am on Oct 7, 2017 (gmt 0)

Moderator This Forum from GB 

WebmasterWorld Administrator mack is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:June 15, 2001
posts:7799
votes: 73


Great topic. It always makes sense to try and reduce the reported number of direct traffic as much as possible. I like how you demonstrated how to add values to the URL to make email traffic traceable. I tend to take this a stage further, and just give it a numeric value. It keeps the URL shorter and I can use scripts or analytics to "decode" it.

I am also a fan of using URL shorteners simply for email links because you can gain specific analytics for the clicks. I ended up developing my own URL shortener simply for this purpose.

Mack.
1:22 am on Oct 8, 2017 (gmt 0)

Moderator

WebmasterWorld Administrator ergophobe is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 25, 2002
posts:8627
votes: 274


The nice thing about UTM parameters as opposed to other methods is that, if you're within the Google ecosystem (AdWords, GA), they are handled automatically.

Of course, you can set up filters and you can adjust your view settings to handle any parameter your choose. It just takes a little extra effort, so for someone new to web dev who is drinking from the firehose, it simplifies things substantially. If you use UTM params, then medium, source and campaign automatically end up thrown in the right buckets in your Acquisition reports
1:50 am on Oct 9, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member 5+ Year Member Top Contributors Of The Month

joined:Mar 15, 2013
posts: 1002
votes: 96


In my message boards, I've started to strip any "utm_" from links that users submit. People follow a link from one site, find it interesting enough to post on my site with the utm_ params, then (presumably) when someone clicks through from my site then it falsely inflates the numbers coming from the original website.
9:33 pm on Oct 9, 2017 (gmt 0)

Moderator

WebmasterWorld Administrator ergophobe is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 25, 2002
posts:8627
votes: 274


It is an issue with any URL-based tracking parameter - if people copy and paste it into an email, it can inflate the number of clicks you're getting for that campaign.

On the other hand, I wish everyone would copy and paste *with* the UTM parameter, because that would tell me about additional reach. I would *love* to see thousands of clicks on the spring 2018 newsletter that had the referrer coming through as Twitter and Facebook. That would tell me people were sharing the link.
12:55 am on Oct 10, 2017 (gmt 0)

Administrator

WebmasterWorld Administrator phranque is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Aug 10, 2004
posts:11566
votes: 182


and there's evil url hackers like me...
8:52 am on Oct 10, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member 5+ Year Member Top Contributors Of The Month

joined:Mar 15, 2013
posts:1002
votes: 96


On the other hand, I wish everyone would copy and paste *with* the UTM parameter, because that would tell me about additional reach. I would *love* to see thousands of clicks on the spring 2018 newsletter that had the referrer coming through as Twitter and Facebook. That would tell me people were sharing the link.

But imagine this very likely scenario.

You make a post on Facebook, and one of your followers shares it on my site. Then 1,000 people on my site follow the link with the UTM param showing Facebook. You falsely think that the post on Facebook brought you 1,000 clicks, when it was really the post on my site that did it.

You then spend money marketing on Facebook instead of my site.
7:17 pm on Oct 10, 2017 (gmt 0)

Moderator

WebmasterWorld Administrator ergophobe is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 25, 2002
posts:8627
votes: 274


From your perspective as a publisher, that makes sense of course.

From a marketer perspective, trying to determine the sources of my traffic, which is what we're talking about in this thread, it's an attribution modeling question. From that perspective, Facebook *did* bring me that traffic (at least as a contributory, initial cause) and I *do* want to spend my money on Facebook.

Some people say all attribution models other than last click are garbage. Attribution models that account for prior actions are often used by agencies to inflate their value (esp. with "view through" metrics as opposed to "click through" and "associated revenue" as opposed to actual, trackable transactions). I recently got upset at an agency who was claiming a 26:1 ROI based on view through numbers, which, was total BS.

But as an analyst, I am curious about clicks other than the last click if I can unearth it. Similarly, I would love to know where that blogger discovered my link. So if he pastes it in with the UTM parameter, that might be bad for him (i.e. you in your scenario), but it is good for me when looking at my analytics and understanding my traffic sources.
7:46 pm on Oct 10, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member 5+ Year Member Top Contributors Of The Month

joined:Mar 15, 2013
posts:1002
votes: 96


What you've described would destroy my marketing campaign, so it's interesting to see the other POV :-)

Perhaps there should be a 4th UTM parameter (utm_origin?) so you could see both the origin and where the click actually came from.
7:50 pm on Oct 10, 2017 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12913
votes: 891


I strip out all parameters from the data I process.
8:32 pm on Oct 10, 2017 (gmt 0)

Moderator

WebmasterWorld Administrator ergophobe is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 25, 2002
posts:8627
votes: 274


csdude55 - the thing is, you of course control your own site. Personally, when putting a link on my site, I *always* strip out the UTM parameters just like you do. But as someone analyzing my own site, I wish everyone else would be so kind as to add in my UTM params.

And in fact, in many cases people will, especially with paid listings, because in that case the parameter will flag the link source. So that's the flip side - if you're a publisher who is offering real value to advertisers with paid listings, you should encourage them to add UTM params so they know they value you offer.

>>4th UTM param

Well, there are actually 5 UTM parameters: utm_campaign, utm_source, utm_medium, utm_content and utm_term. So I think you're thinking of a 6th parameter. The thing is, it would never get used - in paid links, you'll be using the five classics and in unpaid links, nobody is going to take the time to add an utm_origin parameter.

I'd settle for getting keyword data back in Google referral strings ;-)
11:41 am on Oct 20, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Nov 15, 2001
posts: 1732
votes: 36


I have started wth UTM codes on bing advertising which is being recorded faithfully now in analytics, but something has gone wrong with my attempt to do it with LinkedIn. I wonder if it is something to do with https on our site.

I wanted to find out what UTM stands for, google tells me it is something like " Urchin Tracked Metric " I used to love Urchin :-)
2:32 am on Oct 22, 2017 (gmt 0)

Moderator

WebmasterWorld Administrator ergophobe is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 25, 2002
posts: 8627
votes: 274


>>Urchin Tracking Metric

Well, according to the co-founder of Urchin

If Urchin 2 got us in the door, and Urchin 3 didnít suck, Urchin 4 was actually pretty respectable. It had the then-Apple-esque brushed-aluminum look, some fancy-for-the-time interface elements, and most importantly, it had the UTM. The UTM, or Urchin Traffic Monitor, was an early method for augmenting Apache (or IIS, etc.) log files with cookies, such that unique visitors could be established.
-- [urchin.biz...]


Google, however, now calls it the Urchin Tracking Module - [support.google.com...]
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members