Welcome to WebmasterWorld Guest from 54.196.98.96

Forum Moderators: Robert Charlton & goodroi

Google Will Eventually Stop Following Links on Noindex Pages

     
3:54 pm on Dec 21, 2017 (gmt 0)

Administrator from GB 

WebmasterWorld Administrator engine is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month Best Post Of The Month

joined:May 9, 2000
posts:25476
votes: 744



In a Google Webmaster hangout, John Mueller said, Google will eventually stop following links from a page that has noindex on it.
You may have to rethink how noindex is used on certain pages.

[youtube.com...]
6:36 pm on Dec 21, 2017 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:15034
votes: 666


I was hoping this would shed light on a question that has occasionally exercised my mind: what happens to images belonging to noindexed pages? If the image doesn't have a noindex header in its own right, the image could theoretically be shown in search results--but would its associated page be linked?
8:14 pm on Dec 21, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member aristotle is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Aug 4, 2008
posts:3410
votes: 279


Not 100% certain, but most likely a noindex tag applies to all the content on the page, including images. But If someone wants to make sure that the images aren't indexed, they can use an x-robots tag in .htaccess:
<Files ~ "\.(gif|jp[eg]|png)$"> 
Header append x-robots-tag "noindex"
</Files>


As for the main topic of this thread, I've used the tag for [noindex,follow] quite a bit on my sites:
<meta name="robots" content="noindex, follow">

But apparently this tag will no longer work when this new policy goes into effect.
9:15 pm on Dec 21, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member aristotle is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Aug 4, 2008
posts:3410
votes: 279


For those who claim that google's algorithm doesn't use nofollow links in its calculations, this announcement should remind them that nofollow links are a "sinkhole" for some of the "link juice" that flows to a page, and therefore their presence on a site actually can affect its rankings.
9:36 pm on Dec 21, 2017 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:15034
votes: 666


content="noindex, follow"
ďfollowĒ in a <robots> meta has always been redundant, since that was the default. Itís ďnofollowĒ that would potentially now also become redundant. Unless, of course, you care about search engines other than G###.
10:06 pm on Dec 21, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member aristotle is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Aug 4, 2008
posts:3410
votes: 279


Yes, "follow" has always been the default, but I've always put it in anyway for emphasis.

At any rate, after this new policy goes into effect, pages that have that tag will absorb all of the incoming link juice and won't re-distribute any of it either internally or externally. This will eliminate one of the SEO options often used for helping to control the flow of link juice.
2:45 am on Dec 22, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member Top Contributors Of The Month

joined:Apr 1, 2016
posts:2044
votes: 529


But apparently this tag will no longer work when this new policy goes into effect.


This isn't a new policy, John is describing the mechanics of how things work and have worked since a long while. Moreover, there is a subtle difference between follow and no-follow when no-index is applied. But it is subtle and in all but a few edge cases, completely useless. What John said was that no-index at first prevents a page from showing in search but does not remove the page from the index, and in this case "follow" will still allow links to be followed. But after a longer period (no mention of how long!) Google will remove the pages from the index, and since they are not in the index, links cannot be followed. So, "follow" become useless.

In summary "follow" with no-index is pointless unless you only "no-index" the page for a short period of time.

Or at least this is how I understood what was said in the hangout.
9:33 am on Dec 22, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Feb 12, 2006
posts:2696
votes: 112


Could be annoying for events sites like mine. At the moment I am noindexing all the date pages that are in the past. But if Iím going to lose all the link juice from their backlinks then Iíll have to index the whole lot again ó or is that how I should be doing it anyway? NickMNS makes it sound like Iím already losing the juice already
9:50 am on Dec 22, 2017 (gmt 0)

Administrator from GB 

WebmasterWorld Administrator engine is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month Best Post Of The Month

joined:May 9, 2000
posts:25476
votes: 744


In summary "follow" with no-index is pointless unless you only "no-index" the page for a short period of time.

I can see a use for longer term noindex. If you run an ecommerce site and your item is replaced by a new product, you don't want the old product appearing in the SERPs as a sales item, but you do want it on your own site, you noindex it. But, you do want the link and follow so that the new product gets indexed. Some older products are supported for many years after the peak sales cycle.
10:12 am on Dec 22, 2017 (gmt 0)

New User from SE 

joined:May 23, 2017
posts:3
votes: 0


What about paginated pages that uses "noindex, follow". That should still tell Google to follow the links on paginated pages, but not indexed them...?
10:37 am on Dec 22, 2017 (gmt 0)

Senior Member from GB 

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Aug 11, 2008
posts:1695
votes: 255


Pagination is now best done via canonical tags, not noindex.
2:43 pm on Dec 22, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member aristotle is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Aug 4, 2008
posts:3410
votes: 279


In summary "follow" with no-index is pointless unless you only "no-index" the page for a short period of time.

That's incorrect. The noindex tag tells google not to show the page in its search results. But followed links on the page will currently still pass link juice. That part won't change until after the new policy goes into effect.
5:51 pm on Dec 22, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member Top Contributors Of The Month

joined:Apr 1, 2016
posts:2044
votes: 529


@aristotle
That's incorrect.

You can argue with me all you want but I'm just relaying my interpretation of what was explained in that video, and the explanation is clear. Furthermore there is no mention of any new policy in the video.

@Williamhollingworth
The question regarding "no-index + follow" was raised exactly in the context of pagination so you might want to check the video and rewind a bit from the bookmarked location, there may be some interesting insights there for you.
10:15 pm on Dec 22, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member aristotle is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Aug 4, 2008
posts:3410
votes: 279


NickMNS

Your entire first post is incorrect. You need to study the matter yourself to improve your understanding. That's the best way to learn.
11:57 pm on Dec 22, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member 5+ Year Member Top Contributors Of The Month

joined:Sept 14, 2011
posts:1045
votes: 132


This is how I see it:
current = follow links from noindex pages
future = ignore noindex completely
5:07 am on Dec 23, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member Top Contributors Of The Month

joined:Apr 1, 2016
posts:2044
votes: 529


You can see it anyway you would like, and you are free to think that I am incorrect, but this is not my opinion, I am simply reporting what Google is saying. Do with it as you please.

Also there is no future state, this isn't an announcement of a change to the algo, it is explanation of how thing currently work.

Engine's title for this thread is terribly misleading and ambiguous.
Google Will Eventually Stop Following Links on Noindex Pages

It is not that at some time in the future (eventually) Google will make a change and start to no longer follow no-indexed links. It is, if a page remains no-indexed for a long enough, then after a certain period of time (eventually) the links will no longer be followed.
12:00 pm on Dec 25, 2017 (gmt 0)

Full Member from ES 

10+ Year Member Top Contributors Of The Month

joined:Jan 20, 2004
posts:345
votes: 24


The biggest use for noindex,follow in a post-Panda world was for archive, tag, category and author pages on Wordpress sites. Most people thought they were solving the problem of indexing old posts without filling G's index with junk, snippet-collection pages. This may have worked in the short term, but, if I understand what I'm reading here, not in the long term. In any case, attention needs now to be focused on how best to index and position (where appropriate) old content.
10:54 pm on Dec 25, 2017 (gmt 0)

Moderator from US 

WebmasterWorld Administrator keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12361
votes: 807


Engine's title for this thread is terribly misleading and ambiguous.
I don't think the thread title is misleading nor ambiguous. I understood it just fine. It is followed up by a link to the Google Webmaster Central office video.
12:10 am on Dec 26, 2017 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:15034
votes: 666


It might be less ambiguous if it were put in the present tense: Google eventually stops following et cetera.

Not that I'm entirely prepared to believe that at any time earlier than heat-death of the universe, Google (or any other search engine) would ever forget it has seen a link.
5:47 pm on Dec 26, 2017 (gmt 0)

Moderator

WebmasterWorld Administrator buckworks is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Dec 9, 2001
posts:5782
votes: 112


without filling G's index with junk


Some of my sites have pages that are necessary for the site but aren't appropriate to be indexed for search. At present I have many pages no-indexed but if that leads to less link juice for the site as a whole I'd have to re-think that.

To me it would seem utterly perverse if sites who take steps to keep weak pages out of the index end up harming themselves by doing so. Something doesn't add up here.
7:29 pm on Dec 28, 2017 (gmt 0)

Junior Member

Top Contributors Of The Month

joined:July 20, 2016
posts:79
votes: 16


I got a site where I offer downloadable contents. I have noindexed all the download pages because if I don't they would start appearing in search engines messing up my site. So it does not make any sense to stop following links on noindex pages. All those sites who host downloadable contents usually noindex the pages where download is automatically initiated or started by clicking download button. From such pages, users go to other pages while file is being downloaded in the background.
8:02 pm on Dec 28, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member aristotle is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Aug 4, 2008
posts:3410
votes: 279


What about pages that DON'T have a noindex tag but google's algorithm still chooses to exclude them from its index because of very low quality, etc?

To be consistent, shouldn't google also start treating all the links on these algorithmically-deindexed pages as nofollow?
9:20 pm on Dec 28, 2017 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:15034
votes: 666


But do those pages with downloadable content--or pages that aren't considered worth indexing--link to pages that aren't reachable by any other links on the site? If so, there's something seriously wrong with the site design.

I don't think g### ever truly excludes a page if there isn't a noindex tag. It may be ranked as no. 15,678 in search results, or hidden behind a "for other results very similar to this one", but it's in there somewhere.
9:50 pm on Dec 28, 2017 (gmt 0)

Moderator from US 

WebmasterWorld Administrator keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12361
votes: 807


John Mueller may have recently said "Google will eventually stop following links from a page that has noindex on it"... but this is not new. It's the way it's been.
2:59 pm on Dec 29, 2017 (gmt 0)

Senior Member from CA 

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Nov 25, 2003
posts:1153
votes: 295


Given my sites architecture: about &^%$#@! time.
However, given that
* I get several Google 'requests' annually telling me that I have noindex on certain pages (and robots.txt blocking said pages - so how do you Google know about the 'backup' meta noindex?) and that it would be beneficial (for whom?) to remove that and allow G to show the pages...
* sneaky Google (and other SE) bots frequently attempting access to noindexed pages, including ones only accessible via links from noindexed pages...
I take John's comment with a dump trailer load of salt. G is too addicted to bring them all and in the darkness bind them to ever actually stop discovery. Especially 'hidden' discovery.
6:22 pm on Dec 29, 2017 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:15034
votes: 666


sneaky Google (and other SE) bots frequently attempting access to noindexed pages, including ones only accessible via links from noindexed pages
How does that qualify as ďsneakyĒ? If they havenít seen it, they canít know itís noindexed--and even if they have been there before, they canít know that itís still noindexed, any more than they can know whether any other content has changed from one crawl to the next.
7:27 am on Dec 31, 2017 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member tangor is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 29, 2005
posts:8581
votes: 674


CIIW, but noindex does not mean the spider won't see the content to know we don't index it. Therefore G already knows all, but this is letting webmasters know that if you actively block it, we'll comply ... with a vengeance.

That said, tempest in a teapot. All should look at how noindex is being used ... and if being used to game the system, expect a pushback.

Or am I missing something here?

Disclaimer: I have zero (0) noindex on any of my sites. Period. I don't want it indexed, I don't post it.
4:56 pm on Dec 31, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member Top Contributors Of The Month

joined:Feb 12, 2006
posts:2696
votes: 112


I have zero (0) noindex on any of my sites. Period. I don't want it indexed, I don't post it.

that's not always an option for some sites... i've got an events site and there's no good reason for old, expired events to appear in google's index, but if i'm going to lose the page juice from the backlinks them then i've decided to index the whole lot. that's what i've done this week -- for posts going back four or five years. none of them are linked internally anymore, but they've appeared back in google's index within a few days.
5:30 pm on Dec 31, 2017 (gmt 0)

Senior Member

WebmasterWorld Senior Member Top Contributors Of The Month

joined:Apr 1, 2016
posts:2044
votes: 529


@londrum
but if i'm going to lose the page juice from the backlinks them then i've decided to index the whole lot

I don't think that this is an optimal solution either. Adding pages that are "low quality", that is that are not worth indexing will consume crawl budget. So Google may well spend a lot of resources crawling and ranking pages (poorly at that) instead of focusing on new potentially high quality content.

Alternatively, had the pages been not no-indexed Google would have drop them from the index anyways. So the links would not have counted regardless.

And now with your current change of states, going from no-indexed to indexed Google may well index and rank the old page and then in medium-term revert back the alternate state, thus simply causing short term flux for no long term gain.
5:47 pm on June 6, 2018 (gmt 0)

Full Member

joined:Dec 11, 2013
posts:237
votes: 40


So what is the consensus on that: is it better to "index,follow" low-value pages to make sure the link juice from those pages is not lost OR it is better to "noindex" low-value pages and lose the link juice?
This 48 message thread spans 2 pages: 48