A Fix for Google, How to Implement Referral Privacy Correctly

[inbound]

I can see the reasons behind not sending referral data that includes the query; there are many places in between Google and the intended website that could harvest that data.

I'm throwing down a challenge to Google. As I would assume that Google are OK for the visited site to have access to the query string (but not those inbetween) I suggest this (and I know that people might not like WMT, but if Google send you traffic you might have to do this to "play ball"):

Implement a system which uses encryption of the query string, with site owners given access to the decryption key (through WMT) that could be used with code available from Google - this key could also be shared with analytics programs to decode the querystring.

This means only Google and the target site can read the querystring, so features on sites that rely on the query being there will be still work (with some updates to the code).

If Google think that giving the querystring to the target website is a problem then I would have to disagree with them entirely - they have the ability to add (if it's not already there) a line or two to their T&C's to allow them to do this without fear of privacy authorities. Passing the querystring is helpful for the consumer in many cases.

Will Google do this?

[tedster]

Implement a system which uses encryption of the query string, with site owners given access to the decryption key

I like this idea a lot. First, it assumes that data privacy is the real issue, as Google publicly said it is. If that really is the case, then Google has a responsibility to find a solution that doesn't eat 10% of the keyword data and this proposal sounds like a start.

However, I still don't feel we've had a clear enough explanation of the problem that needs to be solved by hiding keyword data. Without that, it's pretty hard to know if any alternative really addresses the problem that Google has been working on here since at least last year.

[Leosghost]

Nice idea ..but ..I'm not holding my breath..on Google even replying ..let alone saying yes..

It isn't that their explanation isn't clear ..it is that their explanation doesn't hold up to scrutiny at all ..and that they could have done the SSL part without removing the "query" ..They have tried to put its removal down to technical reasons due to the implementation of their SSL..and that is not true..they chose to remove it ..and not for protect the "privacy" of the user reasons..

[inbound]

I can see why people might think this (or something similar) is unlikely to happen, there's a lot of scepticism about the reasons put forward by Google for the change (and their actions, leaving QS in paid traffic).

But having any workable "solution" available means that Google can choose to implement it (and get a pat on the back for doing it correctly) or choose to ignore it and prove the sceptics correct.

[Maurice]

Good idea - might stop the analytics companies queing up to join in the anti trust hearings

[incrediBILL]

Hardly a fix as the website decoding the data could then pass the decoded referral data to 3rd parties so Chitika, etc. will properly display targeted ads. Since it's those 3rd parties who they are trying to keep the data from in the first place, this suggestion will never fly. The temptation to make money by passing this data to affiliates and other referral programs for targeted advertising is too great.

[inbound]

Hardly a fix as the website decoding the data could then pass the decoded referral data to 3rd parties

Controlling access to the decryption key through WMT would allow Google to enforce terms and conditions on webmasters wishing to use the QS. If there is foul play then Google could change the key and stop the flow of information (or stop sending any encrypted QS). There are issues with any "solution" but there has to be a limit to if's, but's and maybe's.

Incidentally, I can see where you are coming from - it's a matter of trust. If Google does not trust you then you lose some of the advantages thay can provide you. Although this should not happen carte blanche, as it negatively affects the public.

[jecasc]

I use the the information about search queries to improve my websites or ad campaigns like every other webmaster.

But if I am honest I have to admit that - as useful as the data is - we as webmasters have no right to this information - especially as long as it can be attributed to a single visitor.

It is private information that should never have been shared with us in the first place. Not by any search engine. The user submits his query to the search engine and the average user is not even aware that this information is shared with website owners.

The reason we get it is, that historically the search engines have used GET and not POST as method in their forms and so the query strings ended up in the referrer. But this has been a mistake and huge gap in privacy from the beginning.

Actually I can't remember how often I have found embarassing queries from visitors in my logs or in logs of companies I have worked for. "Please god make my #*$!les go away" has been one of the more harmless ones. I have seen everything - from medical conditions to political affiliations. Queries From Herpes to Hepatitis from racist to #*$!ographic. Just do a quick search for "funny search queries" and you will find several thousand webmasters who publish the funniest queries they found in their logs.

The problem is not that a middleman might get this data. The problem is that we as website owners get this data. Always has been. This is the real privacy issue here.

The data usually remains anonymous but everybody here knows that once a user creates an account on a website it is possible to attribut the query to a specific individual. Depending on the type of website you get this information complete with address, phone and credit card number.

When your customer buys Aspirin on your website it is actually none of your business that he searched for "cure against hangover" and if he buys washing gel you should not know that he searched for "getting rid of #*$!les in my butt". And it is none of your business if the Vaseline was intended for cuts in boxing or for other purposes.

So the only correct way to implement referall privacy correctly is to not send the search queries through the referrer at all. To nobody.

The information should only provided as anonymized statistic like in Google Webmastertools.

[TravelMan]

This is an awesome idea - each domain gets a key, Google sets the hash.

Sadly, it's probably easier ( and more profitable) for Google to ride the storm and just put up two fingers to the world.

People talk about single digit numbers today, but we all know how that number could rise considerably.

One important thing to keep front of mind is to not forget that they allow the query string to stand for their advertisers using adwords, which kind of blows any real privacy concern argument right out of the water. Either they're concerned about user privacy or they are not. They can't talk out of both sides of their mouths and expect us to seriously buy it.

[inbound]

The problem is not that a middleman might get this data. The problem is that we as website owners get this data. Always has been. This is the real privacy issue here.

There are arguments to be had about this, and everyone has different sensitivities. I would argue that it's totally fine, but our opinions are less important in this particular instance as Google has chosen specific ways to deal with the situation (different depending on paid or not) and has chosen a way to explain their stance. As long as their stance remains the same it's reasonable to ask them to put a "fix" in place that addresses their concerns.

[wheel]

AFAIK, the referrer should be showing the page the visitor came from, not just the site. And in Google's case, that page includes the search term. By not providing it, they're basically ignoring standards the web has used for years.

And I don't believe for a second this has anything to do with privacy.

[explorador]

This topic witll open a lot of valid discussions, but from the beginning, Google is not doing this because of the privacy issue, not a chance.

While I agree that lots of queries involve private or embarassing data, if this takes place we will be AGAIN guessing what's happening behind the scenes with G, confused, wondering. G has been accused of acting in favor of its own solutions (currently under investigation) and the guesses are based almost in reverse ingeniering. We will lose that too.

[wwconnect]

I don't think there is all that much hypocrisy in Goog hiding the data for organic but providing it for AdWords advertisers.

Goog's whole premise is that searchers know the difference between an advertisement and an organic listing. If this is the case Goog can reasonably argue that there are different expectations of privacy when someone chooses to click a paid ad versus an organic listing.

There is however a strong argument that the distinction between ads and organic listings has almost disappeared.

[wwconnect]

@Inbound - If privacy is the real concern, your concept would also let Goog role out a privacy fix for AdWords using the encrypted key.

[Leosghost]

Goog's whole spin that they'd like us to swallow is that they believe that searchers know the difference between an advertisement barely distinguished as such, and an organic listing.

there ya go ..fixed it for ya :)

[wheel]

All the American advertising advocates got all wild up a few years ago over paid blogs being unethical because it wasn't apparent to visitors that they were paid endorsments. Didn't the US even pass legislation?

But not a peep over Google's display of ads versus organic. yet on my monitor, there is no visible difference between the paid endorsements of #1/2/3 in the listings versus the unpaid spots in ranking #4 and higher. And if ever there was a venue where consumers are clicking on things they think are endorsments (high ranked in Google) but in fact are paid ads, it's the top 3 adwords listings in Google.

[Sgt_Kickaxe]

Implement a system which uses encryption of the query string, with site owners given access to the decryption key (through WMT) that could be used with code available from Google - this key could also be shared with analytics programs to decode the querystring.

In theory I love the idea, in practice it wouldn't make Google a profit and is contrary to their "collect all data about websites/visitors but share as little as possible about Google" mantra.

I can however envision the data being part of the paid analytics platform, I'm sure they want all sources of "incremental revenue" on their balance sheet.

[indyank]

To all here, Why should google provide this data to webmasters?

[indyank]

Advertisers pay them money and they are bound to give them useful reports. But why do they have to give this to others who get "free traffic" from them? Is there any legal binding?

[Leosghost]

To you..read what we already wrote ..and given their past history with regards to privacy like "accidentally" slurping up wifi data with street view..and then denying that they did ..even if said data was publicly available ..( although how one can write a piece of software and then claim not to know what it did )..and then backtracking* and claiming that they didn't know that had it ..the further backtracking* and saying that they didnt notice..than further backtracking* and saying it was "some engineers" oversight /fault..

Why would websmasters be "untrustworthy" with this data ..and Google "trustworthy"..and how does paying for an ad make one "trustworthy"..anyone can buy an ad with Google adwords with a ripped off credit card..and many bad guys do..

btw ..did you know Google allow premium publishers to run adsense on cyberlockers..I know at least one that has adsense on it for over two years..and the adsense is not served by a "third party network"..it is sent direct from G ..

*lying
** lying more
***lying still more

and they are bound to give them useful reports.

no ..they are bound to run the ads providing the click through and tracking and query data is optional, and something they have decided to do )..and who defines useful ?..and they say it is for "privacy"**** reasons...they do not say it is for "commercial" reasons..

****lying even more again.

[edited by: Leosghost at 5:26 pm (utc) on Oct 21, 2011]

[indyank]

It still doesn't explain the question.

The question of "trustworthy" or "untrustworthy", "lying" or more lying" isn't relevant.

I think they have the right to determine how and with whom this be shared. When they are not bound to send you "free traffic" they aren't bound to give you this data, isn't it?

[Leosghost]

You have a very strange way of looking at the world..the free traffic is because people click on the parts of my website(s) that Google reproduce on their pages ..for which I do not charge them to be allowed to gather..

Doing this is breaking and not honoring their side of an unwritten but implied and understood to exist by all parties "covenant" ..

And claiming to do so for reasons of user privacy is dishonest..

[indyank]

I hear you....

But when you have people, organizations and countries who easily break written covenants but still qualify for bonuses, implied covenants doesn't have much value on Earth and no, I am not surprised by this act or have a strange way of looking at it.

[incrediBILL]

Controlling access to the decryption key through WMT would allow Google to enforce terms and conditions on webmasters wishing to use the QS.

Using that theory there wouldn't be any abuse in AdSense yet there is, MFAs and worse all over the place, not to mention click fraud. Google bans them as they go along and more just pop up in their place.

Likewise, AdWords also has fraudulent ads in it, so Google doesn't do as good of a job policing as one would hope.

That's why it won't work, you can't trust the webmaster to maintain privacy.

Plus, how do you prove a webmaster is breaking the privacy?

[ponyboy96]

We all know why this was done specifically to organic results. Google does not like SEOs. It's plain and simple. I happen to know a couple of former G engineers who told me this is the case; although, they say the G won't publicly admit it. They spend millions per year to undo the things that SEOs do to exploit their algo. Why do you think MC is at all the conferences and on the SEO forums? To learn what we do and counter it.

Giving us less data will help diminish SEO. Try getting that increase in budget or your project approved as an SEO without some conversion or traffic data for your targeted keywords. It's hard enough now even with the data, much less without it.

[randle]

How much of an impact will this really have though? My understanding is that if the searcher is not logged into a Google account, nothing will change.

The amount of people searching on Google, who are actually logged into a Google account, has got to be incredibly small. The only reason most people have a Google account is for gmail, and what % of the overall email market is Gmail? (compared to the others?)

Am I missing something here? (usually am so go easy on me)

[Leosghost]

It is the "camels nose"..something that Google use all the time..

[g1smd]

has got to be incredibly small

For some types of sites it will be quite large.

[randle]

For some types of sites it will be quite large.

Good point, I could see that happening for certain subjects. But if you took a snapshot of one entire days worth of searches conducted on Google, I can't imagine it would crack 5%.

[tedster]

The amount of people searching on Google, who are actually logged into a Google account, has got to be incredibly small.

Google said it's about 9% and I recently saw a private, independent study that came up with just over 10%. Not all that small.