Supplemental and Omitted Results

Why do they even exist and why are they so slowly updated?

[zeus]

Ok we all know Google must have some HUGE problems these days, we have NEVER seen so many posts with wierd stuff related to Google the last 1-2 years and all looks to be related to index problems.

Omitted results why?, why not just list all what is in the DB and dont let the user after 4 pages, have to click the link omitted results and then start the search again.

Supplemental results what do we need that for, if they think there is a dublicate page dont index it, but the problem is those "supplemental results" often just have the same meta description as other pages, then they are supplemental, EVEN if the body of the page is a 100% different, also IF by misttake there is included a dub page, then the webmaster has changed it, even a 404, then the supplemental result page is still in the index even if the page dont exist, that is just not a updated search engine.

Look at MSN, Wisenut they dont have those wierd problems and they are up to date with there index everyday.

[asusplay]

The other problem is that once a page goes supplemental it's nigh on impossible for it to be in the main index again and rank for even secondary or minor keywords. And any attempts tro correct this get ignored because Googlebot then pretty much stops visiting supplemental pages with any sort of frequency.

[soapystar]

zeus this is something ive also tried to get my head around. I can see why it would have been more pr friendly if google had tagged some pages premier and left them in the premier index for quality pages and then a normal index. In other words exactly what they are doing now but with different tags. Then you'd be looking at the same situation but from a totally different perspective. Webmasters would then be asking not why is my page tagged supplemental but what can i do to get my page tagged premier. The one thing that throws all logic out of the window currently is the fact that simply having no meta or very similar metas can a throw a page supplemental. If content is king then metas must be emperors i guess.

[g1smd]

The "Omitted Results" are obvious. They hide results that are deemed to be "too similar" to the ones already listed. It even says that in the on-screen message at the end of the listed results. Matt Cutts [threadwatch.org] already made comments about that very recently. It is easy to fix those problems on a site.

As for Supplemental Results, see this other thread: [webmasterworld.com...]

[zeus]

Also with google small bugs here and there, something is a little scary, you can easy destroy another site, with simple tricks.

Make a 302/301 to the other site, sometimes it will get the sites linked to as cache.

copy a few pages from the other site, then it will be supplemental and omitted.

I think I got 400 pages which is supplemental and ALL dont exists anymore(404) and the real version of those pages dont get indexed and once again WHY DO WE NEED THOSE SUPPLEMENTAL RESULTS delete those IF they are realy supplemental (google try look at the body also and maybe if its a image site which dont have that much text) or buy some more servers like other (MSN)

[asusplay]

Zeus has a point. It is said the supplemental pages problem is easy to fix is pages are "too similar". But if the meta description and the actual content of the pages is different from page to page and all that is the same is the template (code) then there's is an issue because it seems Google doesn't look at the content once a page is deemed as "offensive" (i.e gone supplemental). Just my experience.

[Bewenched]

The other problem is that once a page goes supplemental it's nigh on impossible for it to be in the main index again and rank for even secondary or minor keywords.

I'll have to disagree a little bit on this one. We had alot of pages go supplimental due to the site being spidered under SSL (go figure) Anyhow those have been supplimental for 6 months now and they are now starting to pull out of it this week. I'm not sure about how the rankings are doing on those that are coming back in, but I'm hopeful they will do well again soon.

[webdude]

Mmmm. Might be a clue, but I see when using the site: to search, certain pages are listed as supplemental. But try searching various phrases and see what is supplemental and what is not. It seems the supplemental index is fluid and wrapped in the regular SERPs somehow. Example...

site:mysite.com filter=0 shows Promotional Widgets page as being supplemental.

search promotional widgets shows Promtional Widgets page in the normal returns.

Am I missing something here? For me, pages seem to be listed in both depending on the search methodology.

[g1smd]

Any particular URL that returns a "200 OK" status will have an entry in the normal index for the current content, and an entry in the Supplemental Index for the old content. Old content is whatever Google indexed at that URL several weeks or months ago.

So, it depends on what you search for as to whether any particular URL shows up as a Supplemental Result or not.

If there is duplicate content, there might only be an entry in the Supplemenal Index for that URL - and the content will appear in the normal index for some other URL (different dynamic parameters, non-www vs. www, some other related domain, etc).

If a URL now redirects, or is 404, then it will show as a supplemental result for a whole year before Google drops it from view.

[g1smd]

There is also the issue of similar title tags and similar meta descriptions, and this is an issue that the site owner can easily clear up.

.

If you sell a thousand different widgets then Google will not want to list all your pages as:

Cheap Widgets
Buy your cheap widgets from us; we are great.

Cheap Widgets
Buy your cheap widgets from us; we are great.

Cheap Widgets
Buy your cheap widgets from us; we are great.

Cheap Widgets
Buy your cheap widgets from us; we are great.

Cheap Widgets
Buy your cheap widgets from us; we are great.

Cheap Widgets
Buy your cheap widgets from us; we are great.

Cheap Widgets
Buy your cheap widgets from us; we are great.

Cheap Widgets
Buy your cheap widgets from us; we are great.

Cheap Widgets
Buy your cheap widgets from us; we are great.

Cheap Widgets
Buy your cheap widgets from us; we are great.

Cheap Widgets
Buy your cheap widgets from us; we are great.

Cheap Widgets
Buy your cheap widgets from us; we are great.

In what way could SERPs like that ever benefit a user?

.

Instead they will want to list something like this:

Rotating Widget Type 471
Type 471 widget in red or blue from Acme Widget Corp.

Hand-made siver widget (4 pack)
The new limited edition silver widget from Widgets R Us.

Widget holder
Widget holder suitable for all Acme Corp widgets. Bulk discount available.

Widget type 45Z in green
Enjoy 50% off. This is the final batch of the now discontinued type 45Z green widgets. A classic!

Widget type 55G
Type 55G is the long-awaited replacement for type 45Z. Limited quantities in stock. These are selling fast.

Widget Guide
Comparison chart of main widget features for the whole Acme Widget 2006/2007 range.

Widgets from Acme and Widgets R Us
Feature comparision chart between the brand leading Acme and Widgets R Us products. Updated 2006 September.

Widget Order Form
Use this priority order form for next-day shipping to all contiguous US and Canadian states.

Widget History
The widget has only been around for two decades; read here to find out how this great product was designed, and manufactured.

Widgets in the news
Links to some press about these widgets and their benefits and impact on modern society.

Readers Widgets
Find out how our customers are using their widgets. Share great ideas online with other people in your area!

This is Design 101, and SEO 101.

.

Where is the problem?

The problem is on your site, not with Google.

[webdude]

The page in question is unique in both content, title, and description. While searching for the title, it returns in the regular SERPs. While searching using site: - it returns as supplemental. No problem here, just stating a fact. And yes, I agree, it is SEO 101.

[soapystar]

well im glad you said

Google drops it from view

since a lot of people think they actually get removed from the servers.

[zeus]

Its not all site with supplemental results that can be changed, I got a 1-2 year old site which got supplemental because of same meta description, but body content was 93% unique, then I made different description for each page which is TOTALY NUTS because a normal person can see the uniqur content and we may NOT make pages for search engines, but seems lately WE HAVE TO.

I also got another site 6 year old, but it was through the googlebug hell, other sites made 302 links to it and those where cached with our content on the 302 linking domains, that site is totaly stucked, I have created new pages, changed description, everyting nothing helps.

Another site very popular 30.000 unique a day PR6, got partly supplemental because google in some way found a part of the site which i did not know that was double, I have not fixed that bug so those pages come with a 404, but nothing has changed still supplemental.

I think its ok to filter a little dublicated content be cause they dont have the database space, but then they realy have to be double and NOT only the meta description, that totaly nonsens, thats a ban for nothing.

[g1smd]

>> I have not fixed that bug so those pages come with a 404, but nothing has changed still supplemental. <<

URLs that have become 404 and then show as Supplemental will continue to show in the SERPs for a full year before Google clears them away.

[zeus]

g1smd - thats another negative thing about that system, why have 404 pages in the index even if they are supplemental, they still can be found under unique searches.

[g1smd]

So that the page you had online yesterday, but which is gone today can still be found by the searcher that was looking at it yesterday, and wants to see it again.

[texasville]

There is more than one reason for the supplemental issue. Identical metas and titles or whatever is just one.
I recenty built a site for a friend and very much avoided this issue. All unique tags. didn't matter. All but two pages have gone supplemental.
The main reason has to be that the site has very few inbound links and the interior pages probably have none other than the other pages on the site from the menu.
My question is ...as the pages gain ibl's over time..will they come out of supplemental.

[zeus]

g1smd - something like that.

Its incredible I can do what I want with one of my sites, 5 days ago it was supplemental, all of now I have non supplemental, because of meta description and title, but still this sucks, be cause I can also do what I want with another site and nothing happens.

[confuscius]

"Google drops it from view" - in my opininion, this is probably the most important facet to understand about the Google search methodolgy. If you imagine that Google crawls a website and adds what it finds to its massive database of known pages. If we then have an algorithm that at its simplest consists of overalying the underlying dataset with a series of simple filtering rules then what you would see in the search results is not a positive identification of good results but the residual results of excluding the bad results!

Google go to great lengths to point out how they like to "deal with" issues through algo changes - perhaps, that is their only option!

My observations of Google over the past 2 years or so lead me to believe that as Google have had to write more "overlay bad boy rules" they will have written them within the context of not shifting the search results too much and that what we are now seeing is so many rules that they are creatin anomolies that are starting to have undesired effects.

I will give you one example of this methodology in action - in fact, I personally believe it is Google's current way of addressing the subdomain spam issue. So, try this for yourself, pick a word that any self disprespecting spammer might choose to target. Next step, insert a number somewhere after the first letter of the word and before the last letter of the word. Next, run the search. It is highly likely that you will see results 1 to a small number of a large number of results. The filter in operation hides the pages that have been tagged with "autobanned" words. Also, you should find that you can view less than 1 percent of the reported result numbers. However, the results that you do see - see if you can find a supplemental result! Rinse, wash and repeat. In other words certain searches produce only one type of result. It is also possible to produce completely supplemental results, if you know how. As a consequence of this particular exclusion methodology then anyone who scraped content would have a good chance of scraping spam content containing "autobanned" words! A rather neat way to try and control the issue.

All this leads me to my personal number 1 golden SEO rule - if there is any word used on my site which might possibly be construed as not "a normal and accepted usage" then use Google to check its current banned word methodology.

[dede_dublin]

I have 3 sites

First one is dynamic site with 1000's of pages, signed up for site map
did the www - nonwww redirect, wrote the robot text, unique content, unique meta tags, about 70 links and after all my effort gone supplemental (except the index page).

Second one about 20 page html website, all the pages has same meta tags,
2 links, no www 301 redirect, no sitemap, no robots text, no fresh content, left since last year. No supplemental

third one is again 20 page html website, signed up for site map
did the www - nonwww 301 redirect, wrote the robot text, unique content, unique meta tags, about 2 links and all supplemental (except the index page).

I think I made a big mistake by signing to sitemap.

[zeus]

"I think I made a big mistake by signing to sitemap."

Ohh I would NEVER sign up for such thing, why? if you have normal links and a sitemap, then you dont need to ALSO submit your map, be cause it will be spidered anyway, to close contact to a SE is not good they also do not want a to close contact with a webmaster, also google has alot of privacy holes, so would not sign up.

[g1smd]

URLs that you have redirected will continue to show up as a Supplemental Result for one full year after the redirect is put in place.

Don't be fooled into thinking that "nothing is happening". When the year finishes those Supplemental Results get "cleaned up".

[soapystar]

its not just being able to add more urls if you have enough trust..it covers everything...you only have to look at the massive cross linking of some very heavy weight brand leaders...no way could a mom and pop do that...

[zeus]

Where is the LOGIC in having supplemental results listed which are now a 404, I just dont get it.

Also why do some changes to a supplemental result page change to a non supplemental resutls and listed after 3 days, where other pages on another domain nothing happens for month, please tell.

[g1smd]

I assume that Google hangs on to ghosts of pages that are now 404 (and for domains that have expired too) simply so that a surfer who was looking at some page on your site just the other day, and now wants to look at it again, can still do so, even though you have removed the information from your site. Those results for "non-live" pages are just one type of thing that the Supplemental database contains. It also contains the old content previously found at URLs that are now just redirects, and several other things.

>> Why do some changes to a supplemental result page change to a non supplemental results and listed after 3 days, where other pages on another domain nothing happens for month, please tell. <<

Many factors are involved with that. However, you will notice that if you make a change to a page that is live and shown as a Supplemental Result that although for some terms the URL will then begin to show up as a normal result, the page will continue to show up as a Supplemental Result for searches involving any words that were in the old content and which do NOT appear in the new version of the page.

Additionally, if the old version of the page is an "exact duplicate" (see my other recent posts for the definitions for that) and shown as Supplemental then sometimes (perhaps often?) the Supplemental Result is not updated at all, but one of the other alternative URLs for that content is the one that gets updated instead.

[zeus]

Today i see a site of mine, which has unique meta, title,body text on EVERY page, gone totaly omitted only the www.domain.com is there, it says 1 of 31 pages but only show 3 with omitted results and why is it omitted its TOTALY different content all over. Damn i slowly hate all those small thing you have to watch to make a good site, you can NOT just concentrate on makeing a good site anymore if you want it to be listed on google, its sick.

[rocco]

zeus - I also have sites like this, but it is an odd behaviour, these omitted pages are not supplemental and can be found for whatever searches. Also, if you login into your gmail account and then use google, you will see all pages and not just 3 of them. Same stands for when using some country specific google TLD's.

[rocco]

btw. those site are pretty young, less than half a year.

[g1smd]

My "yardstick" site (belongs to a friend), with three alternative domains pointing with redirects at one .com site where the content is located. All multiple domain, www & non-www, and index page, problems solved with redirects more than a year ago. All title tag, and meta description content is unique per page. The 70 Supplemental .co.uk listings (indexed before the redirects were fitted) were dropped out of the index a few months ago.

One year ago, various pages from each of the four domains were listed in the SERPs. Most were Supplemental, many were URL-only. Many also had duplicate or missing meta description tags. The redirects were put in at that time, and many (but not all) of the meta descriptions were fixed. The last dozen or so were fixed only a few months ago.

.

A site:domain.com search shows 184 entries:

- 168 www entries in the normal index, all of which are "200 OK". There are 168 pages on the site. Correct!

- 14 www entries showing as Supplemental - these are all pages that have been "404" for a couple of months, as they were moved to a different folder. Their new version, at the new URLs, all show in the normal index, part of the 168 above.

- 2 non-www URLs showing as Supplemental. These reappeared in the listings last week - right out of left field; having originally dropped out of the index more than 9 months ago. As they are Supplemental, and have been 301 for a year, they can safely be ignored.

The first Supplemental Result is 164th in the listings, just ahead of the last couple of normal listings.

.

A site:wwwdomain.com -inurl:www search shows 17 Supplemental entries:

- 14 www entries showing as Supplemental - as above. They are all "404" pages.

- 2 non-www URLs showing as Supplemental - as above. They are both "301" pages.

- 1 www URL that showed up as a Normal Result in the first search now appears as a Supplemental Result in this search. This is a "historical" supplemental. The snippet represents older content from many months ago, even though the cache date is only two weeks ago.

This is the only one that needs checking back on. One link to it from another site might fix the problem, so we already did that.

.

This "update" / "refresh" - no major changes to report; just those two Supplemental non-www URLs reappearing again. This site and several others like it, proved to me beyond all reasonable doubt that:
- the 301 redirects for duplicates of all types are vital.
- unique title tag and meta description data is vital.
- Supplemental Results for 301 and 404 URLs are dropped after a year, not sooner.

[zeus]

g1smd i realy respect the knowledge you have got in this supplemental/omitted results or lets say database saving trick from google, but you must admit not always there is logic to this, sometimes it looks like a lotto ticket if you get troubles and PLEASE we webmasters have other things to do as to comply with google hickups.