Welcome to WebmasterWorld Guest from 18.206.168.65

Forum Moderators: Robert Charlton & goodroi

URLs for Certain KWs Getting Completely Omitted from SERPs

     
6:50 am on Oct 7, 2018 (gmt 0)

Junior Member

Top Contributors Of The Month

joined:Aug 9, 2017
posts:96
votes: 11


Hey everyone,

For the past 1.5 years, we've been noticing that some of our URLs are getting completely omitted from search results for particular KW variations. Looking over at Search console, they do appear to be indexed properly, and nothing holding them back. Additionally, the canonical is specified. If that URL has a pagination sequence (triggered as a result of user-generated content), along with sorting parameters, everything is set up in accordance to Google's documentation, as well as to the advice of professional thought leaders in the industry. For certain URLs, we rank just fine, and it fluctuates within reason, just as it does for anybody. But then for some reason many URLs just completely get omitted from results. Occasionally, they'll flicker back and forth, some stay, some go, and for some, it's a never-ending cycle of coming back then disappearing.

In certain instances when a particular URL gets omitted, it later returns to the SERPs under a different URL of the pagination sequence. For example, https://example.com/topic/ would go MIA but later return to rank much lower but under https://example.com/topic?page=5

At first, the issue wasn't too big, but over the past 1.5 I am beginning to notice more and more URLs are particular query variations just get omitted. After each major Google update, from which we weren't negatively affected, I notice this issue get “magnified” and spread. again, for the URLs that actually rank, they rank strongly, but some random ones get omitted. The code structure is identical all across the board and we can't seem to detect any particular pattern within the pages that are disappearing from the SERPs.

In fact, we've also noticed that when we publish new content they never appear in the SERPs, although everything indicates that they're properly indexed. Occasionally, they appear on the first page for a day or so, and then vanish again.

We've investigated over and over, and no one, including us, can seem to identify a root issue. Any help here would be very much appreciated as this is beginning to negatively affect us as this issue is spreading across more URLs. Thanks!
4:10 pm on Oct 15, 2018 (gmt 0)

Full Member

5+ Year Member

joined:Dec 11, 2013
posts:269
votes: 52


I'd suggest:

- no sorting parameter in prev/next rel,
- meta=noindex on sorting parameter pages,
- rel=nofollow on links containing the sorting parameter.
2:00 am on Oct 16, 2018 (gmt 0)

New User

joined:Oct 11, 2018
posts: 1
votes: 1


So how is it possible to noindex pages such as page1, page2 without using robots.txt file. I see Yoast does without the use of robots.txt file.

My website is getting serious penalization due to the indexes of page1, page2, page9 etc. But I don't want to use noindex for these pages in robots.txt file.
2:29 am on Oct 16, 2018 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member keyplyr is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Sept 26, 2001
posts:12913
votes: 891


wildweb - there are 3 ways to use noindex:

• domain level robots.txt
• page level meta tags
• directory level X-Robots headers

[support.google.com...]
3:18 am on Oct 16, 2018 (gmt 0)

Moderator from US 

WebmasterWorld Administrator martinibuster is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Apr 13, 2002
posts:14886
votes: 484


My website is getting serious penalization due to the indexes of page1, page2, page9 etc...


Do you feel that's the case because those pages are thin?

Or because they are exact duplicates of other pages?

Or because the content is partially duplicated and mixed and matched with duplicate content from elsewhere around the site?

Do you have a sitemap that's auto-generated? You might want to check if there are pages listed in the site map that are truly thin, with almost no content.
5:38 am on Oct 16, 2018 (gmt 0)

Junior Member

Top Contributors Of The Month

joined:Aug 9, 2017
posts:96
votes: 11


In our case, since we have canonical setup for each page in the pagination sequence as the user is browsing through consumer reviews (the canonical will match the URL of the pagination sequence that user is currently on, if we're on page 6, the canonical will be www.domain.com/product-review?page=6), we sometimes notice that google displays the wrong URL for a particular KW in search. So instead of the main root canonical (www.domain.com/product-review, it will display the url for example www.domain.com/product-review?page=6). And since page 2 and beyond are just customer reviews without the actual product review authored by us, we rank lower everytime Google is displaying the wrong canonical.

Sometimes if we return back for the root main canonical, we rank on top, but if we disappear, we sometimes will appear on page 3 for the wrong canonical. Very frustrating.
This 35 message thread spans 2 pages: 35