SSL and HTTPS/HTTP content duplication

[followgreg]

Hi,

I added SSL to one of our sites back a few months ago.
I was thinking that Google should be smart enough to understand that's the same site (unlike with subdomains).

Apparently not, they picked up the httpS and it's been close to a catastrophe since then.

So, SSL really causes duplication?

[turkey]

I have read just about everything about http and https for the past year. I have also tried and learned by trial and error alot. Infact as soon as it was suggested that https would give you a slight serp boost that was my next move. What a waste of money it is.

To be honest it did not perform in serps as well, it was said to be. But even though I still have it I switched back to http, and boom. Now I am ranking 1 - 2 position in the toughest keywords, higher than ever before back to http.

It could be that even though my website had 301 from http to https, I kept on building backlinks for http (thinking what would happen one day if I decided not to keep the https, then all the efforts would go to waste.

Well there are statements that the link juice transfer slightly falls back a bit, when you 301 to https, true. Then there are issues in Google WMT that will make you wonder what the heck is going on.

In Google WMT you have to set up 4 website versions,

http://
http://www.
https://
https://www.

then select the domain you prefer or else Google will see your website as duplicate content and serps will drop until you fix it.

As Google even see the www and non-www version of your website as 2 different domains.

[edited by: Robert_Charlton at 2:57 am (utc) on May 25, 2015]
[edit reason] Removed autolinking [/edit]

[seoskunk]

You could use a different robots.txt file to disallow the https version of the site:

in htaccess

RewriteEngine On

RewriteCond %{HTTPS} on
RewriteRule ^robots\.txt$ robots_ssl.txt [L,NC]

in robots_ssl.txt
User-agent: *
Disallow: /


This will sort out any duplicate content problems with the site mirrored on https

[followgreg]

The thing is, I have no idea if there is content duplication, all I see is that adding more security seems to have a negative impact (way to go Google!).
I guess that I'll redirect from https to http....honestly I find it totally ridiculous that Google won't transfer the juice from http to https, for subdomains (including www) I understand but for SSL I don't.

[Robert Charlton]

followgreg, I posted some comments in a discussion last August, which might help now...

Does SSL version constitute duplicate content?
Aug, 2014
https://www.webmasterworld.com/google/4693714.htm [webmasterworld.com]

I'm not sure from your post and question whether you understand that when changing an http site to https, you would have to 301 redirect all of your urls from http to https for Google properly to handle the change.

You only want one protocol to resolve for each individual page. Both protocols returning a 200 OK header response is dupe content, and https/http dupes can be a mess to clean up.

Also, regarding...

robots_ssl.txt

For a bunch of reasons, I wouldn't use a robots.txt or robots_ssl.txt approach to clear up the problem. Whether it works will depend on how your server and site are set up. In any event, though, ultimately you're going to want to use mod_rewrite for a lot of other canonicalization that it appears your haven't done, so you might as well tackle it now. FYI, this thread covers some of the issues with robots_ssl.txt...

How do I serve a different robots.txt for https:
Is it possible to serve up a different robots.txt file for https
https://www.webmasterworld.com/apache/3936164.htm [webmasterworld.com]

[not2easy]

I added SSL to one of our sites back a few months ago.
I was thinking that Google should be smart enough to understand that's the same site (unlike with subdomains).

Either you want to use SSL (https) or not. If you added it to get the benefit, the entire site should be https now, you should have added the "new" domain to your GWT (or "Search Console"..) account and submitted a sitemap for the https site.

As Robert says, it needs to be 301 redirected to the https version or it will be seen as a new site that is a duplicate of your existing site. You can tell Google if you wish, but if you have 301 redirected to the new URLs, you won't need to because they won't be able to access the old URLs. So if you can access your site with both http: and https: you missed an important step. There is help for rewriting URLs via .htaccess in the Apache Forum: [webmasterworld.com...]