Huge number of supplementals went into regular index "overnight"!

2006-06-12: New Update or Filter Tweak?

[Marval]

Noticed starting yesterday morning there is a huge shuffle in the SERPs - of course its impossible to gauge by the old indicators, but I think we are seeing a "change" (I hesitate to name it an update as Ill probably get shot down) - only concrete evidence Ive seen so far is huge amount of formerly supplementals joining the regular index almost overnight

[guru5571]

Very positive for me too. Thought, I'd mention it here since the Data Center Watch thread seems to be where the casualties go to lick their wounds and discuss various flavours of fecal matter.

[Scott_F]

Ok, well now I am seeing totally different results now (3rd different set today) and love them. I hope they can stay like this.

[g1smd]

... and what of those old supplemental results that I was very pleased to see Google throw away about a month ago, after having them hanging around for years and years?

Almost a half of them have reappeared back in the index overnight. Pfffffft.

[Trisha]

g1smd - have you heard if Google is treating 410 pages any different now? When doing a site: search some of my 410's are showing as 410's in the results, rather than being in supplemental with the old content from them still being in the results.

But now I wonder, will Google treat all the 410'd pages as duplicates of each other?

Things have been going well for my main site for a while now, and I thought it was due to the extra work I was putting into it - but today with this tweak or whatever - its all gone.

[g1smd]

Matt Cutts has said, on several occasions, that they treat 404 and 410 the same.

>> some of my 410's are showing as 410's in the results <<

What do you mean by "showing"?

[stinkfoot]

lol now this is what i call weird ... I went looking for a 4 word search term in google and low and bwehold .. it brought up x million results ALL supplimental.

Perhaps i was looking at the datacentre that had the (sup results) flag turned to all on and not like your datacentre which seems to be (all off)

Gotta love google what a pile of trash it really is

Actually i wonder if they are just hiding the truth of the matter now ... not telling us what is and what isnt in the sup to keep us quiet about how terrible this thing is as a search engine. Reminds me in fact of something they used to call PR, now it is pretty much an irrealvance as they dont show you anything like the "truth" of the matter.

[Trisha]

I thought maybe there was a chance they had starting treating 410's differently now.

Like when I do [site:mysite.com] the SERPs show this in the results for a number of pages:

410 Gone
Gone The requested resource /shp/xx/xx/xx is no longer available on this server and there is no forwarding address. ...
www.mysite.com/shp/xx/xx/xx - 1k - Supplemental Result - Cached - Similar pages

So it would appear that my site has a number of pages with the title '410 Gone'. With the same content on them 'Gone
The requested resource xx is no longer available on this server and there is no forwarding address. Please remove all references to this resource.'

I'm really grasping at straws here. Every few months for a couple of years now this site loses almost entirely all of its Google referrals. Then a few months later they magically come back regardless of whether I do anything or not.

This time I wondering if the supplementals could still be causing a problem, possibly as duplicate content.

[g1smd]

OK, you have a page that says "410 Gone" in the text returned for that URL, but what is the HTTP Response Code that is returned for the URL?

I assume that the server is not returning a proper 410 response in the HTTP Header, but instead it is returning a 302 or a 200 response or something.

[Trisha]

I thought it was a real 410, how do you tell the difference? Can I sticky you a URL to see if its a realy 410 or not?

[montefin]

Continuing to emerge from SERPlemental situation:

Now back up to 43% Regular SERPs, 57% SERPlementals.

[g1smd]

Use a HTTP Header Checker to see the HTTP Response Code.

There are several such online checkers, or try WebBug with the HTTP/1.1 setting.

[Trisha]

I used the WebmasterWorld one and got:

HTTP/1.1 410 Gone
Date: Sat, 17 Jun 2006 00:40:11 GMT
Server: Apache/1.3.36 (Unix) PHP/4.4.2 mod_throttle/3.1.2 FrontPage/5.0.2.2623 mod_psoft_traffic/0.1 mod_ssl/2.8.26 OpenSSL/0.9.7a
Connection: close
Content-Type: text/html; charset=iso-8859-1

so I guess its ok then?

(don't know why the date is wrong though)

[egurr]

G still is clearly having problems. One site we worked on is just a local small business. The site went supp, a few weeks ago and hasn't come back. The are #4 on MSN and second page of Yahoo. This is not a site that generates revenue, no ads, nothing but content.
Flashing back to Eric Schmidt's statement about a storage crisis. Looks like they're just pulling random sites at this point.

[g1smd]

>> don't know why the date is wrong though <<

If you did the check just one minute before posting here, then both the date and the time are 100% correct. Your server responds using the GMT (UTC) time zone.

[adamovic]

Seems like an major index update.

[youfoundjake]

72.14.207.104 is showing better results for my site
50 pages listed (i have 100 total) and their index is growing everyday. About 40 of them are supplemental.
I'm pretty fond of that particular DC. Im just glad that Im getting my tiny site indexed again, even though Im changing everything right now and optimizing it.

[jrs_66]

My indexed pages have just jumped from ~300 to my full 2000. Non supp pages have increased from about 70 to ~250. This seems to be spreading quickly on many DC's. Happy fathers day to me!

[g1smd]

Are your Supplemental results mainly for www pages that no longer exist, or for non-www URLs that have had a 301 redirect on them for a while, or for something else?

[jrs_66]

My supps are entirely due to 'lack of PR'- according to Matt Cutts. I posted on his site a while back and got this response-

'There’s absolutely no penalties at all on your site; it’s jut a matter of PageRank in your case. You’ve got good links right now, and several hundred pages crawled, but a few more good links like you’ve got now would help some more.'

[kidder]

Over the last 24 hours we are seeing big jumps in page counts - even on one of our sites that we expect never to see again in google...It picked up 900 pages in 24 hours. Another has picked up 400 overnight and still climbing.

[rkhare]

finally my site seems to be out of supplemental problem, and indexed pages are now increasing at good pace.

[g1smd]

Check multiple datacentres, it is likely that some have very different results.

[montefin]

montefin wrote:

"Continuing to emerge from SERPlemental situation:
Now back up to 43% Regular SERPs, 57% SERPlementals."

Losing ground again: today 14% Regular, 86% SERPlemental.

[jrs_66]

montefin,

Same basic thing happened to me... google heartburn continues...

[bwnbwn]

took a look at a search in my counter from the keword that brought in the hit everthing was indexed in may of 2005 all the sites or most of the sites were down I got a hit off the search but that poor person was wondering what the heck is going on with most of the first page showing can't display pages or parked domains....

[montefin]

Even worse, our best performing page which was "vanished" back in April during the Google Sitemaps robots.txt "issue"...

...and which surfaced again high in the SERPs (see earlier in this thread)...

...is "vanished" again.

[kidder]

In the last 5 days we hav gone from 500 pages in the google index to 15,000 and adding new content pages daily. It's all looking like it's working again. The new content we have been adding is in but not ranking just yet. 2 months ago we were at 10k in google so it's a roller coaster. We did add a blog to the site and some new html pages - changed the link structure slightly.

[lorenzinho2]

A few days ago I posted about how starting June 17, our very deep pages were no longer generating Google referrals. The vast majority of these pages are in non-commercial, non-competitive subjects.

Upon further investigation it looks like we've had thousands of these deep (and legit) pages, with unique content, go supplemental.

The flip flopping of supplemental pages would seem central as to what's going on with this update.

[The_Tank]

As of this morning I am now seeing many more pages being listed under the site: command. Up from 400 to 5800 - which is a bit closer to my real site figure. However as with others these pages are marked supplemental.

I think with all the crap from Google that our site has had since April that this actually counts as a small step in the right direction.