Huge number of supplementals went into regular index "overnight"!

2006-06-12: New Update or Filter Tweak?

[Marval]

Noticed starting yesterday morning there is a huge shuffle in the SERPs - of course its impossible to gauge by the old indicators, but I think we are seeing a "change" (I hesitate to name it an update as Ill probably get shot down) - only concrete evidence Ive seen so far is huge amount of formerly supplementals joining the regular index almost overnight

[tedster]

I saw something like this on one site I work with. Sometime in the last 48 hours or so over 500,000 Supplementals went into the regular index. I did nothing to make them go Supplemental, and nothing to bring them back.

Darned if I know what's going on. Some of those urls SHOULD be Supplemental -- or even gone completely!

[webdevfv]

Hurrah (of sorts)

I hadn't had a new product page indexed for almost a year.

Overnight I've had pretty much all new product and category pages added to the index - except for ones built in the last week or so.

Unfortunately, they are all listed as SUPPLEMENTAL. Oh well, can't have everything. Just hoping that when it settles they get added properly - or am I being unrealistic?

[phantombookman]

72.14.207.104
The above is showing several of my sites with greatly increased counts, also looking like accurate numbers!

A lot of them are supplemental but at least they are there.
My blog looks a mess though, they are showing comments in the index but not the posts etc.
Hopefully the above DC is a sign of things to come and the road to recovery for the big G

[JoeSinkwitz]

Something is clearly wrong with their indexing right now; one of the sites I manage went from having nearly every product indexed to just one non-supplemental indexing (the root), and now has a few popping in from the supplementals that we deleted in early 2005...wow.

After a rough night of DCing though, I'll take one fixed issue at a time for now.

For those that believe the machine crisis wasn't real...well, given all the strange problems, I don't know how to go with that stance any more. We can be assured that Google really does care enough to fix their problems, but whether or not they have the resources to do so expediently remains to be seen. GG, good luck with that new giant football field center -- I want one too!

[sunny_kat]

It might be a mere datacenter fluctuation. my website status

13-6-2006 - 1 Page in index and 5 supplemental
14-6-2006 - 19 Pages Indexed and 2 supplemental
15-6-2006 - 1 Page in index and 5 supplemental ( Back to the same )

Whats really going on with G ( Have they gone crazy? )

[smokeybarnable]

my site just went from having only my domain indexed to having all my pages indexed...but it still shows the pages as supplemental? I didn't realize they indexed supplemental pages?

[sunny_kat]

smokeybarnable,
Thats happening with many of my sites too.
I feel that the google server limitations problem which happened some days back is still not rectified

[Vadim]

Unfortunately, they are all listed as SUPPLEMENTAL

In my case also the number of the pages tripled but all increase is supplemental.

It seems that Google has decided to show the dropped pages in the supplemental index. It more corresponds to the reality. They are in the Google data bases but are not important enough to be in the main index.

It is interesting that the supplemental pages seem indeed not developed, i.e. Google has no time for them. I see these supplemental pages when search for my site as the whole and don't see when search for the site directories, where they are in.

Vadim.

[Marval]

A quick look this morning shows an increase of pages showing up in the SERPs has increased by a factor of at least 3 - searches done on common words that had 600 million results two days ago are now over 1.8 billion pages - I would say we are seeing a true update based on the accompanying SERP shuffling that is occuring - numerous major sites have dissapeared from the index - at least in the top 100 or so

[pokerjoker]

For these new SERPS, a "site:" query shows 10,000 additional pages that are (at least they should be) disallowed in our robots.txt. Obviously something is broken at the plex and hopefully they are working on a fix.

[sid560]

All our city and state pages disappeared from SERPs overnite. Only indx page showing.

Pages still have PR and show when doing a site search.

Hopefully from reading what is going on here, it may be a google problem not a sandbox. I hope.

[jrs_66]

Google is not broke. According to Matt Cutts, this supplemental/deindexing issue is all by design. Google is now censoring the web. Google su$#s!

[quixote]

My main site, which hasn't been able to get new pages indexed in several months finally had most of my new stuff added to the index in the past day or so. Still having problems with Supp results but the new pages are non-supp and most importantly are INDEXED!

I think the Goog may be getting closer to righting themselves. At least I hope so...

[pemba76]

hi guys,
i am new member here. well i have a website some 2 to 3 months old. till yesterday, other than the home page, 2 other pages showed in the G main index. Today morning when i checked the G Index i saw 1700+ pages shown indexed but all as Supplemental Result. Even 1 of the pages out of the 2 has been moved to Supplemental Index. I am very worried because of the G Supplemental Index.

[trinorthlighting]

Check data center 72.14.207.104

Fresh new cache dates for the supplemental pages.

The site:mysite.com is not working on this center though. Do a search for "mysite.com"

Also, links are getting updated on that data center as well I noticed...

[asiaseo]

I just checked that one for our own site and 2 sites that I am monitoring for clients, new pages done in the last month or so added, not many, highest was 20, all relevant information about new venues or up comming events, every single one that they have now managed to crawl has gone straight supplemental. These are different hosts, even hosted in different countries, but every page staight supplemental.

[rkhare]

hmmm, most of my pages are back now but with supp tag.

[g1smd]

Remember that there are several types of Supplemental Results.

For a page that goes 404 or the domain expires, Google keeps a copy of the very last version of the page that they saw, as a Supplemental Result and show it in the index when the number of other pages returned is low. The cached copy will be quite old.

For a normal site, the current version of the page should be in the normal index, and the previous version of the page is held in the Supplemental index.

If you use search terms that match the current content, then you see that current content in the title and snippet, in the cache, and on the live page.

If you search for terms that were only on the old version of the page, then you see those old search terms in the title and snippet, even though they are not in the cache, nor found on the live page. That result will be marke as Supplemental.

There are also supplemental results where the result is for duplicate content of whatever Google considers to be the "main" site. These results seemingly hang around forever, with an old cache, a cache that often no longer reflects what is really on the page right now. Usually there is no "normal" result for that duplicate URL - just the old Supplemental, based on the old data. On the other hand, the "main" URL will usually have both a normal result and a Supplemental result.

.

Right now I see some interesting bugs in the Supplemental logic.

site:domain.com inurl:www brings 98000 www pages all with a recent cache.

site:domain.com -inurl:www brings 24000 www pages (even though the search says to exclude all www pages) all of them marked as Supplemental and all showing a cache date of almost a year ago.

That should not be happening.

.

Add to that the pages with meta noindex tags on them that have been indexed and cached, and are showing as Supplemental Results with a cache from 2005 June or July, and Google has a bit of a problem on their hands right now.

.

Oh, and searches with a hyphen in them are not fixed either. Search for an email address with a hyphen in it. See what results you get. Search again replacing the hyphen with a space and see that thousands of supplemental pages appear from nowhere - all for pages that have (or had) the email address printed on them.

[montefin]

Not huge here, but certainly a step in the right direction.

Pre-14JUN 10% Regular, 90% Supplemental. Post-14JUN 20% Regular, 80% Supplemental.

But more significant, the best performing page that "vanished" 11-12APR with the robots.txt issue, has reappeared in Google SERPs, position 1.

Hurrah.

[g1smd]

I see a lot of Supplemental Results for old pages, that I though they had thrown away foever last month, are now back in the index on some DCs (72.14.207.x etc).

Noooooooo! Get rid of them.

[daveVk]

site:domain.com -inurl:www brings 24000 www pages (even though the search says to exclude all www pages) all of them marked as Supplemental and all showing a cache date of almost a year ago.

That should not be happening.

My understanding is that the site: term will includes Supplementals while any other terms dont(unless few results).

Hence All-www(nonSups)= www(Sups)+ AnyNonWWW

[pemba76]

just checked my domain using google.com and DC 72.14.207.104 and found no difference.
First i checked at www.google.com:
site:www.domain.com (showed 1,790 and it includes 3 pages in Main Index and 1,787 pages in Supplement Index)
site:www.domain.com - inurl:www (showed 1,000 pages under Supplement Index and no pages in the Main Index! Where has the other 787 pages disappeared?)

Then i checked in DC 72.14.207.104
Same outcome as in www.google.com

I guess all my pages without www has been pushed to the Supplemental Index (but then where is the 787 pages gone - If not in Supplemtal Index then where?).

[Blueshadow]

If google is censoring the web with the supplemental thing than it is not properly done because my website and many others have all the original contents and non or most are not controversial either.

[Marval]

Well - this morning even more pages added to the index - that search term that brought 600 mill results before is now up to 2.3 billion results - totally amazing - noted also that the index includes (in the en-english version) pages that were relagated in the past to country specific searches such as aol.de customers homepages - and sitting at #1 in the SERPs for very high dollar terms and phrases

[rden17]

about a week ago, I had a lot of pages get indexed and SERP were awsome overnight. got good traffic. yesterday, the 15th, NOTHING.

I checked my logs and searched for terms that were ranking well a couple days ago (top 5 in serps) and now they don't even show up. doing a site:domain.com I now notice that more than 1/2 of my pages are no longer indexed - not even supplemental. Nothing controversial about my site at all. What the H@ll happened with google?

google bot has been visiting me on a regular basis.

[Snappy_fish]

Just to report that my site has also completely disappeared overnight for a search term that has been ranking constantly well for the past 18 months. I have been buying links - but slowly and on a bit-by-bit basis. Other than that my site is clean with totally origional content. The ranking for the rest of my not-so-important keywords remains unchanged. Strange - I've not experienced anything like this for any of my sites since Florida (2003). Something happend last night!

[Scott_F]

One of my client's sites as well lost positioning for their main two terms, which for the last two years were on page one. Now they are nowhere to be found. I really hope this is a bug.

[stakaman]

This sounds more like an update to me

[drall]

For my sectors there is no question some form of update going on starting 48 hours ago. All positive so far for us.