Welcome to WebmasterWorld Guest from 18.208.187.169

Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

Google Updates and SERP Changes - September 2017

     
10:45 am on Sep 1, 2017 (gmt 0)

Moderator from US 

WebmasterWorld Administrator martinibuster is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 13, 2002
posts:14935
votes: 494



System: The following 4 messages were cut out of thread at: https://www.webmasterworld.com/google/4860963.htm [webmasterworld.com] by robert_charlton - 3:25 am on Sep 2, 2017 (PDT -8)


How do you compare your pages with those in the top 10?


User intent.
6:19 pm on Sept 19, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member Top Contributors Of The Month

joined:Jan 19, 2017
posts: 670
votes: 246


It did seem strange to me that so much of my recent traffic increase was from direct traffic, and so little of it was from Google. I guess it makes sense if a lot of these visitors are logged into Google and the referrer is not being passed along. It would also explain (I think) why both my Google and direct traffic dropped by roughly the same amounts when I was hit by Google updates in March and May. I could never understand why a Google update affected my direct traffic.
8:39 pm on Sept 19, 2017 (gmt 0)

New User

joined:Sept 19, 2017
posts:2
votes: 0


Site utterly tanked today and is now ranking/bouncing all over creation. Anyone seeing other weirdness today? Strange bird...
8:43 pm on Sept 19, 2017 (gmt 0)

Junior Member

10+ Year Member

joined:Nov 13, 2006
posts:111
votes: 0


Yes on weirdness. I see some sort of rollback. A page in cache is newer than the page being displayed in search results.
8:53 pm on Sept 19, 2017 (gmt 0)

Preferred Member

5+ Year Member

joined:Jan 10, 2012
posts:508
votes: 29


Seeing a lot of movement. Wonder if it's related to the update/pullback over the weekend. Could be in for a big one overnight into tomorrow.
10:20 pm on Sept 19, 2017 (gmt 0)

Preferred Member

10+ Year Member Top Contributors Of The Month

joined:Apr 15, 2004
posts:605
votes: 113


All I see is zombies behaviour has changed, I think they are trying to look more natural and creating less gaps like we are some sort of stupid human beings...
12:00 am on Sept 20, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member Top Contributors Of The Month

joined:Jan 19, 2017
posts: 670
votes: 246


If I could ask another question regarding Google traffic and Analytics reports... Earlier it was mentioned that when someone visits a non-https site while logged into their Google account, the referrer is not passed along. This causes Google Analytics to report it as direct traffic, not Google traffic, even if they're visiting from a Google search. If that's true, and Google Analytics is reporting an unknown amount of Google traffic as direct traffic, then does that mean non-https sites like mine have no way of accurately knowing how much traffic they receive from Google? Is Google Analytics useless for non-https sites when it comes to reporting traffic sources?
12:08 am on Sept 20, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member Top Contributors Of The Month

joined:Apr 1, 2016
posts:2715
votes: 823


Again I'm not 100% certain about this so it would be helpful if someone more knowledgeable could chime in. But, my understanding is that referrer is not included in the http request so it is not that Google Analytics is blocking the information. The information is not passed along at all so you would not even know from your raw server logs.
12:55 am on Sept 20, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member aristotle is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Aug 4, 2008
posts:3661
votes: 374


Only the newest browsers (and not all of them) follow the "rule" about not revealing the referer when following a link from an https site to an http site. Also, I've never heard that being logged into google has anything to do with it, and doubt that it is true.
1:12 am on Sept 20, 2017 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12913
votes: 893


Only the newest browsers (and not all of them) follow the "rule" about not revealing the referer when following a link from an https site to an http site.
True. It also has to do with how the server is config'd, on the HTTPS site. The fields you see in the access log reports is set up that way either by OS default or by the admin.

I haven't used Google Analytics for years, and many things have changed, but if the browser is not supporting referrer, then it won't be in any report, no matter if you're signed into your Google account or not. If it ain't there, it ain't there :)
1:34 am on Sept 20, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member Top Contributors Of The Month

joined:Apr 1, 2016
posts:2715
votes: 823


Thanks Keyplyr and Aristotle
Around here we say: "Je vais me coucher moins niaiseux ce soir"
Translated: "I will go to bed less stupid tonight!"
4:46 am on Sept 20, 2017 (gmt 0)

Full Member

Top Contributors Of The Month

joined:Apr 2, 2016
posts: 343
votes: 59


The true referral data have been missing from Google for quite some time. They just give you sample data. I thought that was common understanding as it has been like that for a long time.
5:01 am on Sept 20, 2017 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12913
votes: 893


@Maximum44 - yes that's been the case for several years from Google properties, but not all data comes from Google.
7:25 am on Sept 20, 2017 (gmt 0)

Senior Member from GB 

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Aug 11, 2008
posts:1717
votes: 263


It's not so much about being logged in to Google, rather that Google moved it's properties to https.

As others have said, it's not necessarily implemented by the browsers correctly, but the referrer MUST be excluded in a HTTPS to HTTP referral.
The Referer field has the potential to reveal information about the request context or browsing history of the user, which is a privacy concern if the referring resource's identifier reveals personal information (such as an account name) or a resource that is supposed to be confidential (such as behind a firewall or internal to a secured service). Most general-purpose user agents do not send the Referer header field when the referring resource is a local "file" or "data" URI. A user agent MUST NOT send a Referer header field in an unsecured HTTP request if the referring page was received with a secure protocol.
RFC7231 Section 5.5.2 (Standards Track RFC) [tools.ietf.org]
[Emphasis Mine]
9:39 am on Sept 20, 2017 (gmt 0)

Senior Member from GB 

WebmasterWorld Senior Member redbar is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Oct 14, 2013
posts:3332
votes: 548


Oh, I haved logged-in this morning to a whole swathe of sites with 0 PVs after 10.5 hours

This is getting more than ridiculous, what the heck is G manipulating now?
10:48 am on Sept 20, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member aristotle is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Aug 4, 2008
posts:3661
votes: 374


As others have said, it's not necessarily implemented by the browsers correctly, but the referrer MUST be excluded in a HTTPS to HTTP referral.

Shaddows -- Hasn't that specification been around for a good while, yet a lot of people are still using browsers that don't adhere to it.

And as more and more people upgrade their browsers, less and less referal information on visits to http sites will be available. So the problem will only get worse for opwners of those sites.

But for referals from google, you can still find exact information in GSC, at least for the main keywords
11:21 am on Sept 20, 2017 (gmt 0)

Senior Member from GB 

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Aug 11, 2008
posts:1717
votes: 263


So the problem will only get worse for opwners of those sites
Indeed. As most high-status sites go https, anyone left on http will start to lose referrer data from a wider set of referrers. This will seriously undermine all analytics packages, including GA.

GSC will not help with non-Google https-sourced traffic.

(Also note that the referring server can set preferences for referral strings, including stripping data. I have absolutely no idea how strictly browsers support those preferences. You can find a horribly-presented explanation here [w3c.github.io])
11:53 am on Sept 20, 2017 (gmt 0)

Senior Member from GB 

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Nov 15, 2001
posts: 1836
votes: 66


So can I get this right. An https page will not send referrer information to a non https page? Will it send referrer information to a page on a different site that is https? or are we all going to have to get used to a lot less referrer information from now on?
12:06 pm on Sept 20, 2017 (gmt 0)

Senior Member from GB 

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Aug 11, 2008
posts:1717
votes: 263


An https page will not send referrer information to a non https page?
A compliant browser will not, no.
Will it send referrer information to a page on a different site that is https?
The host site can set preferences which may be supported by a browser. See here:
[w3c.github.io...]
are we all going to have to get used to a lot less referrer information from now on?
"All" - possibly. HTTP-only* - definitely.

*I'm not sure if a link directed at http, which is then 301 to https will retain the referrer. I would assume not- in which case, everyone will lose data on historic links.
12:35 pm on Sept 20, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member aristotle is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Aug 4, 2008
posts:3661
votes: 374


I'm not sure if a link directed at http, which is then 301 to https will retain the referrer. I would assume not

That's a good question -- I hadn't thought of it.

Most likely a browser can be programmed to carry referal data through a 301, but this doesn't mean that it will be done.
12:38 pm on Sept 20, 2017 (gmt 0)

New User

joined:Sept 20, 2017
posts: 1
votes: 0


Been working on a new site with little authority for a few months now and we were just starting to see some good results in SERPs. Whatever happened last night has destroyed several months of hard work. Look forward to hearing from others with similar experience.
1:22 pm on Sept 20, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member Top Contributors Of The Month

joined:Jan 19, 2017
posts: 670
votes: 246


So it seems, based on the discussion, that Google Analytics is less than accurate when it comes to reporting traffic sources. What about overall traffic numbers? Is it less than accurate there as well?

And are there any tools similar to GA that give more accurate results? Or is GA considered "good enough" for the average site?
1:59 pm on Sept 20, 2017 (gmt 0)

Full Member

10+ Year Member Top Contributors Of The Month

joined:Jan 3, 2004
posts:321
votes: 39


Site utterly tanked today and is now ranking/bouncing all over creation. Anyone seeing other weirdness today? Strange bird...


Google has been de-indexing blog networks early in Sept, you are probably feeling the effect. I personally know of about 300 sites getting the axe from 2 different networks.

The ripple effect will carry on for a while
2:04 pm on Sept 20, 2017 (gmt 0)

Junior Member from US 

Top Contributors Of The Month

joined:Mar 2, 2016
posts: 99
votes: 18


Again I see zombie traffic. Low conversion + high traffic flow.

Someone else?

Tourism, USA.
2:06 pm on Sept 20, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member Top Contributors Of The Month

joined:Jan 19, 2017
posts: 670
votes: 246


Other than the panic from waking up to a long list of email notifications this morning from this thread (which turned out to be a discussion about http referrers), I'm not seeing any signs of an update today. Traffic is down slightly this morning compared to yesterday, but nothing to indicate an update. Not yet anyway.

I haven't really felt the effects of any updates since the big one in May. Traffic is still down roughly 18% from before the May update, but at least it has recovered from the terribly slow summer.
2:12 pm on Sept 20, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member aristotle is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Aug 4, 2008
posts:3661
votes: 374


And are there any tools similar to GA that give more accurate results?


GSC provides full and accurate information about referals from google. This is because it gets its information from internal google data bases of search activity. No other tool has access to this information.

Similarly, bing webmaster tools provides full and accurate information about referals from bing

For non-google and non-bing traffic, no tool provides full information.
2:42 pm on Sept 20, 2017 (gmt 0)

Senior Member from GB 

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Aug 11, 2008
posts:1717
votes: 263


For non-google and non-bing traffic, no tool provides full information.
And, cannot provide full information.

All data is ultimately provided by the browser. If the browser isn't telling, there is nothing any analytics package can do about it.
2:55 pm on Sept 20, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member Top Contributors Of The Month

joined:Jan 19, 2017
posts: 670
votes: 246


But what about overall traffic numbers? Is Google Analytics accurately reporting the amount of traffic you receive, even if it can't always tell you where it came from?
3:10 pm on Sept 20, 2017 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member Top Contributors Of The Month

joined:Nov 2, 2014
posts:745
votes: 430


But what about overall traffic numbers? Is Google Analytics accurately reporting the amount of traffic you receive, even if it can't always tell you where it came from?

It can't be accurate since so many ad blockers and anti-virus solutions block the GA javascript from firing.
3:16 pm on Sept 20, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member Top Contributors Of The Month

joined:Jan 19, 2017
posts: 670
votes: 246


Interesting. So there's really no way for any website to know with 100% accuracy how much traffic it receives? In my case, I'm not sure if I would really benefit from knowing the exact numbers anyway. Google Analytics at least lets me keep track of my site's general performance, which is all I really need.
3:18 pm on Sept 20, 2017 (gmt 0)

Senior Member from GB 

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Aug 11, 2008
posts:1717
votes: 263


We trialled Piwik, but prefer GA on web analytics.

Piwik can analyse your server logs if you don't want to interrogate them yourself.

Web Analytics of any type run scripts. I block scripts. I could make a strong guess on some others here who also block scripts. Anyone who block scripts (or makes creative use of a hosts file) will defeat analytics programs.

You cannot defeat server logs, in any meaningful sense.

So, GA will not capture everything, but it will capture >99% of real users and exclude most bots. There is no web analytics that can improve that proposition, though others may have a view on execution.

Other than that, you have server logs, which are basically Reality.

Edit - I see glakes was much quicker than me!
This 363 message thread spans 13 pages: 363