Data Center Watch 2006-05-18

[g1smd]

Continued from:
[webmasterworld.com...]

---

I have been away for a week (just a quick look in, a couple of times during the week, even though I was supposed to be on "holiday") and while I was away Google got rid of the results that I previously referred to as BigDaddy A and B leaving only the experimental results, and the cleaned up version (with the oldest supplementals deleted) left behind, and the "cleaned up" version is the one that is on the vast majority of the datacentres now.

The "same snippet for every page" problem has also been fixed.

Sites that installed a 301 redirect before 2005 June no longer show the redirected URL in the SERPs.

Pages that went 404, and sites that went domain expired, before 2005 June no longer show up in the search results.

New Supplemental Results have appeared for any pages that have changed their status or their content at any time since 2005 June. For pages that are gone, the Supplemental Result has a cache of the final version that was online. For pages that have been updated, the Supplemental Result shows the previous content in the snippet, and the normal result shows current content in the snippet. In both cases the cache is usually only a few days or weeks old.

[edited by: Brett_Tabke at 1:36 pm (utc) on May 19, 2006]

[g1smd]

Hmm, some of my cache dates have updated to May 24th, and other datacentres are still stuck at May 11th.

A new page linked only from that page now shows up in all datacentres, so the new content (the link) added on May 21st has been indexed and followed even if the cache copy hasn't been updated to show that newer version of the page.

[ratzmilk]

Is there really any point trying to guess what is happening with Google when the top result listed for the search term Linux Mandrake returns an "Under Construction' page.

When a default 'Under Construction' page can become the number one page for a search that includes 'Linux', then obviously any attempt at SEO is utterly useless.

I mean, how many pages on the net have the word Linux on them? More than a billion I would guess. And yet, out of all these pages, an 'Under Construction' page that does not have the word Linux anywhere is selected by the algo as the authoritive page.

Come on Matt or GoogleGuy, tell us again how we should design our sites for visitors if we want good search results. Spew forth some more rubbish about Relevance. How long are you two going to keep banging the Relevance drum anyway.

[g1smd]

You've done the search for "Liar" or "Miserable Failure" and understood how those number 1 results happened, and the significance of them, no?

[Atomic]

Does "Miserable Failure" bring up Big Daddy now?

[tebrino]

I'm still confused with what happened on 16th and 17th. Only on those two days Google was showing correct number of pages with my home page included (now it's gone). Would be interesting to hear some official explanataion...

[reseller]

As to search quality (within the sector I watch and my test search query) I see the following DCs show "acceptable quality" serps:

64.233.167.99
64.233.167.104

64.233.187.99
64.233.187.104

72.14.203.99
72.14.203.104

72.14.207.99
72.14.207.104

[g1smd]

I see the May 24th cache dates updated to May 25th on some datacentres now. All of the other datacentres are still stuck on May 11th for the exact same page (and it has the exact same ID in the cache URL).

[frakilk]

64.233.167.99
64.233.167.104

64.233.187.99
64.233.187.104

72.14.203.99
72.14.203.104

72.14.207.99
72.14.207.104

I mean what's the deal with the authority site overload on those DCs. It seems that whenever Google realises their normal results are beyond poor they can crank up the authority switch to sweep the dust under the carpet, so to speak.

[ratzmilk]

g1smd said " You've done the search for "Liar" or "Miserable Failure" and understood how those number 1 results happened, and the significance of them, no?"

Yes I have, that is my point.

There are so many examples of these now. They completely negate everything that is said about on page relevant quality information. At first they were a novilty, now the seem to be the <insert hat colour> SEO du'jour.

I do have a question though, can a googlebomb avoid the sandbox effect?

[steveb]

A large amount of previously never seen supplementals have just been introduced into the index. Apparently "refreshing" supplementals means weirdly adding back more deleted pages from last June.

Note to Google: "refreshing" does not mean digging a corpse out of the ground.

[steveb]

Ugh, even worse, supplementals from 2004 are back on at least 72.14.209.104

[g1smd]

As far as I can tell, Google deleted all of the Supplemental Results from before (older than) 2005 June, just a few weeks ago. Those were often pages that had gone 404, or the domain had expired. They no longer show up at all anywhere.

They also then made a bunch of new Supplemental Results for any page that had changed status since 2005 June.

For any page that has gone 404, or where the domain has expired, since 2005 June, the new Supplemental Result shows the very last version of the page that was online.

For any page edited since 2005 June, and that is still online, the new version of the page is shown as a normal result, and the previous version of the content shows as a Supplemental Result dated anywhere from 2005 June to 2006 March.

Google used to hold on to old data for up to 3 years. It seems now that they only keep it for 9 to 12 months. I see that as an improvement.

[steveb]

...and as I said supplementals going back to at least December 2004 can now be found again, so the pre-June 2005 statement is inaccurate.

[g1smd]

I cannot find anything earlier than 2005-06-20 anywhere that I look.

I do see that at least one group of datacentres has had their index updated with recently spidered data (obtained in last 10 days) today.

[steveb]

As mentioned in the other thread, look with hyphens. Also look -www even where all pages are www.

[g1smd]

Hmm, for a 180 page site with a NON-hyphenated domain, all pages listed as www for the last year or two, with the 301 redirect in place for a very long time.

A site:domain.com search shows all pages as www and all as normal results. Same result for site:www.domain.com searches.

A site:domain.com -www search shows a mix of www and non-www pages, all 20 of which are tagged as Supplemental Results. Some of those Supplemental Results have dates as far back as 2005 October and some are as recent as one week ago. A site:domain.com -inurl:www search shows the same result. The inurl operator is surely broken here.

I don't know why any of these pages are tagged as Supplemental in this search, when they are normal results in a site: search. I don't know why some (just a few) pages also show as non-www (maybe the host took the 301 redirect down briefly, last October?).

I still haven't found any Supplemental Results older than 2005-06-20 but I will keep on looking.

.

OK I partly see what is going on. The 20 Supplemental Results are mostly for pages that no longer exist. Those are the ones with a cache date from 2005 October, and none of those appear when you do a simple site:domain.com search. Still no idea why a few show as non-www though.

However, there are several Supplemental entries mentioned (in the -www and -inurl:www searches) that are for pages that do still exist. Those have a cache date from only a week or so ago. Those are a total anomaly.

[g1smd]

OK. Just noticed that both the [site:domain.com/ -www] and the [site:domain.com/ -inurl:www] searches return zero results as expected.

Additionally, I see that [site:domain.com] returns 180 results and [site:domain.com/] returns 160 results. Hmmm.

[steveb]

Putting the trainling slash does apear to hide the supplementals on the hyphenated domains, which unfortunately suggests that no supplementals actually have been deleted, but are merely hidden in more than one way, with some now revealed by certain search formats.

"Refreshing" these rotting corpses just makes me tingly all over...

[bumpaw]

Why is this? site:www.mydomain.com searched in my default Google page gives me 4 indexed. The same search entered in the McDar tool shows all DC with 480 pages supplemental. It's been like that for over a week.

[g1smd]

Hmm, in the datacentres that had the May 11th cache dates those have updated to May 24th now. For the other datacentres that already had the May 25th cache dates, those have now moved back to May 24th date. All datacentres now show May 24th. This is the cache for one particular page.

Search results on some datacentres (and this number is declining all the time) still find the page for the old content that was removed from the page about 2 weeks ago (about May 15th I think). Other datacentres do not return it for those searches (and that result is spreading to more and more datacentres).

[youfoundjake]

Well, after loosing my entire site to just having the index page listed, I noticed tonight that I'm starting to crawl back in. Not a very big site, 100 pages, but to lose 99 of them hurt. I'm back up to 17 in the past 24 hours which is ok. Been taking this time working on another site that will have 5 times the content. Through this all, I've learned alot about SEO and thanks for the bad robots forum I found last night in the library. Copy pasted the list to my .htaccess. I think Google was just tired, they have been running for 8 years, time to shut down and restart. Its alot of work keeping track of something as big as the Internet. I wonder how long it would take to do a restore?

[petehall]

I now see the 'good results' on the following DCs:
66.102.7.99 (on/off)
64.233.167.99
64.233.167.104
64.233.167.147
64.233.179.99
64.233.179.104
64.233.179.107

First time I've ever seen them on a 66.102 IP I think.

[Seowatch_uk]

Hi Pete something is happenning even this dc 64.233.187.104

[petehall]

I now see it at 66.102.7.104 too, however not sure if my original post was incorrect.

I don't see anything different on 64.233.187.99 / 64.233.187.104 - well that's not true as they have been unique for weeks and weeks now.

I haven't seen any movement on the above 2 DCs for quite soem time though.

[g1smd]

Hmm. More wierdness.

A keyword1 keyword2 keyword3 search shows just 10 results, and is the correct result.

The "quoted" version of that search, as in keyword1 "keyword2 keyword3" drops the 8 Supplemental Results and leaves just 2 normal results showing.

[tedster]

I remember that Matt Cutts on a radio interview, mentioned that the initial roll-out of Big Daddy would be using a Supplemental index that was independent -- "orthagonal" was his exact word -- and not a direct part of the new infrastructure. He said that a new Supplemental index would be integrated into Big Daddy only over time. Perhaps some of this flopping around is the beginning of that integration.

[g1smd]

Yeah, seems like that whatever Matt Cutts says, it always takes 6 to 12 months for anyone to work out what he was actually saying...

[F_Rose]

The datacenters mentioned above are full of supplemental results.

I almost became excited when seeing 520 pages indexed for our site when doing a site: search. But then I noticed that 496 pages are supplemental results..

To me this is far from improvement..

[nippi]

I'm seeing the new results spreading everywhere

[Blade3]

64.233.167.99
64.233.167.104

64.233.187.99
64.233.187.104

72.14.203.99
72.14.203.104

72.14.207.99
72.14.207.104

Out of the above datacenters, I observed datacenter 72.14.207.104 showing the results with the latest date 28 May 2006.