Not Provided - Looks like organic keyword data is gone

[netmeg]

All Google searches, logged in or not, appear to be routed to https today. Clients who previously were running about 40 to 50% not provided are now up above 95%. Looks like organic keyword data is gone.

[goodroi]

I am seeing mixed results from today's data. Some of my sites have lost most of their keyword data but some other sites still have 40-50% of keyword data.

[aakk9999]

I am seeing the same. I am not able to force non-https google search any more although I am seeing a few google searches via http trickling in.

[netmeg]

Danny's post on it:

In the past month, Google quietly made a change aimed at encrypting all search activity — except for clicks on ads. Google says this has been done to provide “extra protection” for searchers, and the company may be aiming to block NSA spying activity. Possibly, it’s a move to increase ad sales. Or both. Welcome to the confusing world of Google secure search [searchengineland.com...]

[JD_Toims]

At 81% not provided today.

[bumpski]

Routers and therefore your ISP and intermediaries will no longer be able to "sniff" the content of the "packets" between your browser and Google. This change blocks all sorts of intermediary collection of statistics and keywords about your activities. If you've tried a "trace route" (tracert), you can see all the rest stops your packets take on their merry way.

It wasn't long ago ISP's talked about inserting their own ads onto any webpages served to users through their services. Using https/ssl blocks this (mostly).

Perhaps some competitive analysis services will lose their data sources.

As far as I have read, the NSA already has agreements, and therefore hooks, into encrypted communications. So this change may actually benefit the NSA and degrade the news media's abilities to collect information.

I looked through my logs and the first request from Google for https pages was about 2/23/2012, and the request rate from Google has now built up to what seems as high as the non-https requests to sites. Literally millions of sites now probably have duplicate content due to Google's indexing https pages.
My webhost(s) and the conglomerate that owns these hosts seems to serve all websites on http and https by default. Of course this is a different issue.

Just some speculation.

[mcskoufis]

From Google's POV this is something that aims to reduce the number of "dull" or "null" searches for keyword rankings. IMHO we won't be able to find which keywords sent us the most traffic, so we won't start to check rankings on them on a daily basis...

Ok maybe this is done for privacy reasons as well, the NSA, etc.. Doubt the NSA won't find a way in though, despite Google's efforts.

On the other hand, this clearly benefits their Ad Platform, agree with Danny on this..

[mrengine]

Keyword data is Google's and Google's only. If they continued to provide it then there might be challenges to their throne. And we know how well Google likes to protect their kingdom from any type of competition. Keep the minions confused and guessing so that they are easier to control!

[jebernier]

So, all we have to go on is the keyword data in Google Webmaster Tools. Wonder when they will be removing that also?

[engine]

What a great way to encourage more marketers to use Google's Webmaster Tools.

This will help kill off independent site stat analysis tools.

[lucy24]

we won't be able to find which keywords sent us the most traffic

... and can't act on that knowledge to make our pages better. ("Oh, oops, didn't realize this page came up in searches for 'widgets in French Polynesia'. Let me tweak the wording so my visitors and I are both happier.")

[Dymero]

Our not provided numbers have doubled in the last month. Looks like the Organic report in Analytics is going to look barren soon.

[netmeg]

Well, there's always AdWords.

[ducks]

[FranticFish]

I always thought this was going to happen but I'm still upset now it has. Looking back at (not provided) over the past two years on a number of sites, it looks to me like this was phased in from mid-July onwards.

I bet Google will tell us this is about privacy. They're big on that aren't they?

Maybe this will finally prompt a class action lawsuit against Google from other stats providers. I'd contribute.

[Robert Charlton]

I bet Google will tell us this is about privacy.

There's part of me that wants to say "privacy", my a**".... and another part that is thinking that there are some privacy aspects, but with Google keeping the data harvest away from competitors and for itself and advertisers... so that is a kind of privacy, but not evenly applied, with loopholes necessarily left for AdWords to function.

bumpski's post interesting wrt who else gets locked out of this information, besides us. Other intermediaries too.

In this earlier discussion, note, per the Danny Sullivan article referenced, the handling of SSL on mobile and the stripping out of referrer data there. Conceivably, the data that wasn't stripped out was already getting pretty much skewed....

Keywords "not provided" trend and consequences
Mar 10, 2013
http://www.webmasterworld.com/google/4553265.htm [webmasterworld.com]


And... does anyone remember wa-a-ay back, when Google tested Ajax-powered serps?...

Google AJAX Powered Serps Kills Referals
Feb 3, 2009
http://www.webmasterworld.com/google/3841736.htm [webmasterworld.com]

...I doubt that it is Googles will to break log analyzers and keyword trackers with this test. I think that is a by-product of whatever Ajax implementation google is currently testing.

On the third hand, it would probably drive alot of webmasters to look to AdWords for traffic. hmmm

And... another more recent sign on the horizon that I'd felt was ominous...

Google: Keywords tool will not be available without Adwords account
July 3, 2013
http://www.webmasterworld.com/google/4589828.htm [webmasterworld.com]

Hmmmm.

[Joseuonline]

Keyword data: exit stage left.

I'm also seeing (not provided) for everyone.

[Sgt_Kickaxe]

I like this change, I've grown tired of my competitor creating dozens of pages targeting a keyword phrase I am #1 for in hopes that one of their attempts stick. No keyword data = no phrase keywords to target.

This will help kill off independent site stat analysis tools.

Good riddance, I've been blocking the bots of several of the most popular sites because they do nothing but scope my site and sell the data to others. The fewer ways a competitor has to attempt to figure out what keywords my traffic comes in with the better, I hope they crack down on automated queries next. This is good news for non-seo minded content creators.

[EditorialGuy]

What Sgt_Kickaxe said.

If I have a beef with Google, it isn't about keyword tools. It's about the quality of the search results.

[turbocharged]

This is just another exit ramp to Adwords that Google created on the internet highway.

[blend27]

If the site goes HTTPS fully, I think the Keyword Data is still passed though. I might be wrong, its been a while.

For me it would be easy, since I write my own code, including keyword analitics and pages visited, real time too :), just flip the switch in validation to allow access to the site only thru HTTPS and redirect all crawl-able content to be crawled via HTTPS protocol only.

Google: Keywords tool will not be available without Adwords account
July 3, 2013
[webmasterworld.com...]
@Brett

I honestly don't think G is shortsighted enough to do something like that system wide for the sake of obfuscating seo efforts. Websites would have little incentive to look to Google for traffic or optimize for Google. The focus would no longer be on Google for optimization. We wouldn't know what or how to optimize for keywords - optimization would be shots in the dark. Our only option would be to look for other big sources of traffic [webmasterworld.com].

Well here we are Ladies and Gentleman.

[blend27]

My main concern would be scrapers here. I already see the uptick in requests coming from hosting ranges & data-centers with a fake referrer of https:// www.google.com/ :(

[indyank]

Doubt the NSA won't find a way in though, despite Google's efforts.

hey come on, this is done to keep others out and not the NSA...it is laughable if you thought google is trying to deny NSA...

[smallcompany]

targeting a keyword phrase

Tired, too. Trying to think useful, but those keep me too busy and off the road.

P.S.
Still would not mind knowing what brings folks onto my sites.

[brotherhood of LAN]

I'm sure Google will see an uptick in automated searches too. It's near impossible to know what you rank for, but some like to keep an eye on important terms.

> If the site goes HTTPS fully, I think the Keyword Data is still passed though. I might be wrong, its been a while.

That's what I thought. Last I heard you don't... because of the way Google redirects.

[Dinkar]

Learn from your ENEMY (google)... if you aren't doing already; start hiding your data NOW!

Anyone who is using google products, shouldn't come here and cry. Either stop using them or milk it as much as you can.

[JD_Toims]

That's what I thought. Last I heard you don't... because of the way Google redirects.

I think that's accurate -- If you right-click on a link rather than looking at the status bar you [I do when I've tried it any way] get a totally different URL than the status bar shows as the destination. In fact, there are no keywords sent to the redirect URL, so there's nothing except what appears to be an encryption for a browser to pass along, even to an https site.

[Zivush]

This is another indication that all Google products are connected - Analytics, G. Search, Adwords/Adsense, Chrome and WMT.
Data is driven, collected and monitored from one source to another. It's their decision whether to share any data or not. There's a good sense in controlling KW data as it reveals how their algorithm works.

[newsnshop]

SEO will be 80% dead without this..

[Tonearm]

Will the keyword data be removed from Google Analytics too? I would think Google could keep it in the know.

[edited by: phranque at 8:59 am (utc) on Sep 25, 2013]
[edit reason] turn off email notification [/edit]

[engine]

This is nothing to do with NSA, it's to do with marketing.

@Sgt_Kickaxe

Good riddance,

You misunderstand. This is not about others harvesting, it's about protected stats for owned sites. If I own the site, and protect my stats (which I do), what's the point of stats for keywords! Keyword data from independent analysis is now gone.

You will have to use Google's ecosystem if you want to know your keyword referrals from Google, and you'll only get the keyword data that Google wants to share.

It's their search engine, so they can do what they like, and they have.

We'll just have to find other independent ways to get the data, or use more of Google's system.

At some point, Google will have pulled in so much into it's own territory, spoon-feeding what it likes, that the competition authorities will have to start looking more closely. Think Microsoft Windows and IE.