How long for Google to re-index to HTTPS?

[Sally Stitts]

I completed ALL my HTTPS prep tasks (link changes and new sitemap) on July 6, 2017, and 301ed everything.
Today, Aug. 16, Google has switched about 50% of my 450 pages to HTTPS in their index (40 days).
Is this about normal?
The rate of changing is slowing down. I would think it would speed up. Silly me.

I had read that it takes from weeks to months.
Am I in good shape? (not to worry?)

Thank you.
.

[tangor]

Will eventually happen. Might speed things along with "fetch as google" to force some updates. Also make sure there are no NON-https entities on the pages.

[Sally Stitts]

Thank you, tangor.

I carefully checked all my pages for any http self links, knowing it would gum up the works.

Now I am going to do a little fetching, especially on some top pages which haven't been reindexed yet.
I am getting a little anxious, since my AdSense has been "kicking in" nicely lately. Effect of going SSL? We'll see.

[not2easy]

You have added the "new" site in GSC, right? You should be able to view the old site's Index Status under the Google Index on the left menu in GSC. Compare that with the old total and the "new" site's Index Status and you should see all pages accounted for. If the "old" site's dashboard still shows any data, you should try to find out what resources are still showing up there. Sometimes old 301s or old site files like privacy policy or favicon were entered somewhere in old pages and are still accessible there.

You don't mention how you 301ed everything. By any chance if you only edited the old www/non www canonical rewrites, there's a chance that some old resources (especially if they are in directories other than root) are not being redirected. I forgot about an old p3p.xml file on my first site that went to SSL. The older the site, the more likely to have scraps around. If you want to use "fetch as Google", do both Desktop and Smartphone and it will list any files it can't access. If you get a 'partial' result and follow their >> to the right you can see those missing files.

[keyplyr]

Hi Sally Stitts,

Related discussion:
What Will Happen if I Don't Switch to HTTPS? [webmasterworld.com]

I've switched over 2 dozen sites to date. The ones that used a sitemap.xml got all the pages reindexed faster (sometimes much faster) than the sites that did not use a site map file.

Obviously this depends on how many pages are on the site and how many pages were in the index prior to the HTTPS switch.

[Wilburforce]

Am I in good shape? (not to worry?)

Yes, that isn't out of the ordinary.

You might also look at your server logs, if you haven't already, and make sure you aren't getting any 200 responses for non-https pages or resources, as that might slow things up Also, have a look at 301s: linking sites may take much longer than Google to notice and update, and if there are a couple of important ones it might be worth dropping them an email.

[Sally Stitts]

I thought I would bump this thing, to add some interesting info.
Almost exactly at 50% now, and stopped cold in its tracks. Updates every 7 days, but the last update showed NO CHANGE at all.

Why? Don't know, but here is HOW.
My top 4 pages, in terms of inbound links (over half), have NOT been recognized as https yet.
One of those pages, provides 50% of all my traffic. Of course, that page is the one I would most like to see as https.
It has approximately 25K incoming links.
Any guesses as to how long they must ruminate over my best pages, before converting them as well?

Any idea of why they take SO LONG to reindex the most popular pages as https?
Must they take a lot of time to reindex pages that have MANY incoming links?

My conclusion? THE BEST PAGES TAKE THE LONGEST TO CONVERT. Just the opposite of what I would like to see.

The "fetch as Google" did not seem to have any effect.
I checked and rechecked my TWO sitemaps - .txt and .htm - no errors on either.
.

[Wilburforce]

One of those pages, provides 50% of all my traffic. Of course, that page is the one I would most like to see as https.

Have you tried Fetch as Google on that page in GSC (in the Crawl section)? If not, do it.

_{Edit - sorry, I replied before I had reached the last line in your message. I'm surprised this had no effect (although its effect would be diluted if you've done it for all other pages as well).}

[Sally Stitts]

As soon as I see a reply, I stop editing my post, and begin replying. Sorry I "got" you. Bad timing.

I did "Fetch as Google", but only on my best page, that I was trying to get switched (most important).

There may have been another issue, as well.
Mediadotnet ad code MUST BE CHANGED for https. The old code results in no ad appearing. I was slow to do this.
I had to contact mediadotnet to issue NEW CODE for my https pages.
But all fixed now. So, once again, it is "Hurry up and wait". Ha.

Here is a link to the NEW FAQs page at mediadotnet - [control.media.net...]
which addresses the need for new code.

"How do I implement your ads on my SSL web pages?
We do provide SSL compliant ads. As this is a custom implementation, please mail us at [e-mail], and your Account Manager will generate the ad codes for your site."

They converted the code for 35 of my 200 ads, not exactly the ones I would have chosen. I guess you get what you get.
Also, under "How Our Ads Work", they say

"Ads targeted to individual pages."

I WISH. Not for me, not yet. STILL by email request, only. All my pages have numerous, varied subject matter. Typically, they take the ad targeting from another popular page, and apply it to a page with a different subject. For me, this does NOT work, because it is NOT targeted to the page.

Methinks they are going to see a LOT MORE "custom implementations", as just about everybody goes SSL.
.

[vphoner]

I am seeing a drop in rank for certain keywords that I wrote down their positions before the switch to Https. One keyword went from 7 to 12 after the page that ranked was re-indexed, others have dropped substantially as well from August 29th (before https) to today. Some keywords have not changed their rank. None have improved.

Will this ranking drop recover? I did everything right on redirecting 301, no http references on site, everything done by the book. Setup analytics and webmaster tools as well to https.

[keyplyr]

There may have been another issue, as well.
Mediadotnet ad code MUST BE CHANGED for https...I was slow to do this.

As you found out, HTTPS pages with secure content errors (unsecure ads blocked) will not be indexed (since they are not secure) until fixed, so those pages will take extra time.

[Mark_A]

What is the status of this https issue with Google? will non https sites suffer a penalty? and if so when is the shutdown date? We have discussed going to https but haven't yet pressed the trigger.

[keyplyr]

@Mark_A - there is no "issue"

Switch your site to HTTPS correctly and there should be no problems. Make sure all file paths are either HTTPS or without protocol.

I recommend switching your site now and not waiting. You may start to loose traffic when the next phase of browser warnings are implemented in October.

[Wilburforce]

What is the status of this https issue with Google? will non https sites suffer a penalty?

It has been widely reported since 2014 that https sites get a small ranking boost from Google, which taken the other way could be viewed as a penalty for non-https sites. Actual penalties are in my view less likely, and in view of the effect warnings will have, probably unneccesary.

As keyplyr points out, the next step-up in warnings is due in October, which will inevitably put some users off non-https sites. In the long-term this will continue in the shape of more and larger warnings in more browsers and search-engines. Google's October change will apply to sites that have any kind of user input, but in the UK and EU - and probably the US, Canada and Australia too - you are already at risk of breaching data-protection law if you collect personal data without https.

Broadly, there are no good reasons to delay moving to https: it isn't difficult, it isn't expensive, and it won't do any harm at all. The only good sticking with http will do is to give those in your sector who have already moved an advantage.

[Mark_A]

Ok, it does not seem hard or expensive so we will do it. Especially because there could be some ranking effect.

I did understand the warnings were going to show for those asking for CC details without SSL, we don't take online payments, our site just promotes us like an online brochure. Would that have made any difference wrt the warnings?

[keyplyr]

@Mark_A - there are several ongoing or recent discussions about the issues associated with *not* making your pages HTTPS. Here's one of them:
What Will Happen if I Don't Switch to HTTPS? [webmasterworld.com]

This next warning update (in October 2017) will be for pages using any kind of input form, including but not limited to CC info.

However, the next warning update (date currently unknown) is expected to show the warnings for all non-secure pages. My guess would be roughly a year from now.

[Mark_A]

thanks keyplyr I found it a little tricky to find current threads on this subject yesterday.

[Sally Stitts]

Just an update.
I went full HTTPS on July 5, 2017. It is now Oct.2, 2017. So, about 3 months.
I am now at 64% converted to HTTPS.
Sure seems slow to me.
I bought the Screaming Frog, and have been fixing every little thing, but still.
Does this seem rather slow? My rate of conversion is getting slower and slower, as well.
It used to be chunks. Now its onesy, twosy. At least it is still going.
.

[Sally Stitts]

Another SIGNIFICANT update.
My BIGGEST problem seems to have been - NO REDIRECT.
I trusted my host to implement the 301 properly.
After pulling my hair out, I finally checked it - no redirect.

My host fixed it in a few hours. Apparently, it is a little tricky on nginx.
But it is working now. No longer implemented in htaccess, but elsewhere in the maze.
Same as with compression. My htaccess now only contains a custom404 redirect.

How did I test it?
I used Bing, since they have yet to index a SINGLE PAGE of mine as HTTPS.
I entered my search term. Then, I clicked on their SERP result, and I got my page as HTTP.

My advice - CHECK YOUR 301 FUNCTIONALITY.
Don't be an idiot, like me, and wait for divine intervention.

It was fixed on Monday, and today, Thursday, I see a 30-page increase in HTTPS indexing (out of 425).

Apparently, the only other issue I have is, I cannot "Leverage Browser Caching".
My score for everything is 100% - 98%. Always dinged 2% because I don't "Leverage Browser Caching".
I use Google and (test site v) to test this.
But then, (test site v) FAILS to pass its own Caching test, which causes me to discount its results.
But, Google gives me the same result.
I don't think I will ever understand caching.
.

[Sally Stitts]

When I first implemented HTTPS (July), my top page had over 25,000 incoming links.
Now, it has about 6,000. (Other pages lost a similar % of links.)
Was this a terrible cost of not 301 redirecting properly?
Will I ever see any of those links again?
If not, then this has been an extremely painful screw-up.
.

[Wilburforce]

I don't think I will ever understand caching.

You're telling the user's browser to store page resources locally for a time. There are other ways to do it, but a statement like this:

<filesMatch ".(css|jpg|jpeg|png|gif|js|ico|ogg|mp3|wav)$">
Header set Cache-Control "max-age=2592000, public"
</filesMatch>

in your .htaccess file will both make it happen and make speed test warnings go away. The numerical value in the statement is the time in seconds the resource is to be stored (2592000 is 30 days), and the rest of it - you can set whatever file types your site uses that you wish to include - is fairly self-explanatory.

[lucy24]

My host fixed it in a few hours. Apparently, it is a little tricky on nginx.
But it is working now. No longer implemented in htaccess, but elsewhere in the maze.

Is it really called htaccess in nginx too?

I cannot "Leverage Browser Caching".
My score for everything is 100% - 98%. Always dinged 2% because I don't "Leverage Browser Caching".

As with “error” messages in GSC, sometimes you just have to put your fingers in your ears and hum loudly.

[Sally Stitts]

@wilburforce
Yeah. Tried that. Was never able to get the warning to go away.
The more I mess with htaccess, the more issues I seem to create for myself.
Sooo, maybe better to live with 2%, than jam myself up with screwdrivers in the mechanism. Just 2%, so wth.
I am currently going with the "fingers in your ears and hum loudly advice".

@lucy24
I have always had an htaccess file.
Still do, with the changeover to nginx. My host has not said anything to me, and
THEY FIRST tried to implement the 301 in my htaccess file. (Didn't work.)

I am not a software person (was a hardware guy, more concerned with nanosecond race conditions) - I just know a little self-taught early html, which makes me dangerous. Ha.
.

[Wilburforce]

I have always had an htaccess file.

I think that is fairly common, but .htaccess is an Apache configuration file, and probably won't work on IIS or Nginx servers.

Nginx will have an equivalent configuration file (possibly in a location you can't access, in which case you'll need to go back to your host), and its own syntax, but I regret that I don't know the file name, location or syntax myself.

[keyplyr]

If you are using an htaccess file in NGINX... Stop. It’s horrible for performance.

NGINX is designed to be efficient. Adding something like this destroys that. Convert the rules, etc to nginx config: [winginx.com...]

[lucy24]

If you are using an htaccess file in NGINX... Stop. It’s horrible for performance.

At the risk of horribly (as it were) derailing this thread... If nginx doesn't use htaccess, how or why would the server even look for the file, let alone read it? Even apache doesn't recognize htaccess unless the server config file specifically says it can.

[robzilla]

You can't use htaccess files with nginx, but nginx is often combined with Apache (usually to keep things simple, ironically, and compatible with the old setups, very common in shared hosting environments). In a traditional setup, the nginx web server is a reverse proxy that directly handles all static file requests, and forwards everything else to Apache, which would still be able to abide by the rules of any htaccess files.

It's no more or less difficult to set up redirects with nginx than it is with Apache, the syntax is just different.

[keyplyr]

This is assuming that the OP is using htaccess because it is actually config'd by the admin or why else would someone be using it in NGINX.

[Sally Stitts]

Thank you all for the input.
I will be consulting with my host right after I post this.

Here is the entirety of my htaccess file --->
ErrorDocument 404 /custom404page.htm

I tried eliminating my htaccess file, but then I don't get my custom 404 page.
So, I put it back, until this is resolved.
It can't be THAT big a hit, with just one line of code, can it?

Sorry, that I am so uninformed (and uneducated). But I am feeling very close to progress, with such great feedback.
Thank you all.
.

[lucy24]

It can't be THAT big a hit, with just one line of code, can it?

Extrapolating from htaccess within Apache: The issue is probably not so much the filesize itself, or the number of directives it contains, as its bare existence--or even the possibility of existence. On every request, the server first has to go looking for htaccess in every directory along the path.

Then again, if your server is set up to allow htaccess in the first place, it's going to take the time to look for it whether or not it even exists, so how much of a performance hit can there be?

the nginx web server is a reverse proxy that directly handles all static file requests, and forwards everything else to Apache, which would still be able to abide by the rules of any htaccess files.

Does “static file” here mean “files that physically exist”? In an Apache-plus-nginx combo, who handles requests for nonexistent files, including vanilla 404s? Do they get dumped on Apache, which would explain the need for an ErrorDocument directive? And, contrariwise, if the requested file does exist, does the server still have to look for htaccess? Can you deny requests both on the nginx side and on the Apache side? So many questions ...