homepage Welcome to WebmasterWorld Guest from 54.226.235.222
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Google / Google SEO News and Discussion
Forum Library, Charter, Moderators: Robert Charlton & aakk9999 & brotherhood of lan & goodroi

Google SEO News and Discussion Forum

This 210 message thread spans 7 pages: < < 210 ( 1 [2] 3 4 5 6 7 > >     
Supplemental Club: Big Daddy - Part 3
icedowl




msg:733996
 3:21 pm on Mar 9, 2006 (gmt 0)

< continued from: [webmasterworld.com...] >

On one of my searches this morning, I found a site with an expired domain above my site in the results. That just ain't right!

[edited by: tedster at 5:58 am (utc) on Nov. 8, 2006]

 

smokeybarnable




msg:734026
 3:37 am on Mar 10, 2006 (gmt 0)

this is a good lesson about too many people relying on google....im doing ok because people bookmark me.

drall




msg:734027
 3:40 am on Mar 10, 2006 (gmt 0)

Tedster

1. Yes, every page of 1500 original content pages except the PR7 homepage.

2. Yes, the homepage is the only currently non supp page. Every page left in the site: command is supp, either old or out of date or cache of 2 years ago in some cases.

3. No ranking at all, all rank for keywords have been destroyed even for the sitename which is very unique, pr7 is still there and cache date is extremely fresh as always.

This is on just one of our sites, it is a "authority" site and has been online for about 5 years now. All Google traffic is gone as one would expect with every page being gone basically as well. None of our other 20 or so sites have been effected like this and all have little to no seo and most are authority sites in there verticals. None of this makes any sense.

trinorthlighting




msg:734028
 3:41 am on Mar 10, 2006 (gmt 0)

1. All live pages of your site excluding your home page have been dropped in the BD datacenters.

They are slowly dropping off

2. When checking site:mysite.com in a BD datacenter, your home page is the first listing followed by supplemental listings that aren't even live pages in your site? Or once were live listings? Some of the supps I have no idea how google would even find without scouring the root of the public directory in my web server!

No, but there are pages that have been dead for about 6 months now

3. When checking google.com for a keyword you rank high in, it shows up sometimes, hit refresh and it's gone, however when checking the cache, it appears that it is staying fairly fresh and the cache date is increasing each day?

No

colin_h




msg:734029
 3:45 am on Mar 10, 2006 (gmt 0)

After the agony of Jagger last year ... this seems like a walk in the park. Really Google, surely you can do better than that ... come on hurt me some more ;-)

phish




msg:734030
 3:50 am on Mar 10, 2006 (gmt 0)

yep same stuff...

only homepage listed normal on site command, rest are supplemental. they are active pages no 404's. this last pr update my homepage actually fell 1 point lower than my internals (go figure) never saw that before. G traffic =0 . yes i will be out of buisness soon.

btw rich, matt cutts deletes all my posts ;)

frakilk




msg:734031
 3:51 am on Mar 10, 2006 (gmt 0)

>>> Oh, and did anyone else find GG's visit yesterday just a little bit odd? It didn't sound like the old GG, altogether less authoritative and just a little bit too nicey nicey. Just a thought, but could google be playing us again?

GG posts for the first time in months and Matt posts about registering for free on WebmasterWorld. Finkle is Einhorn, say no more.

colin_h




msg:734032
 3:54 am on Mar 10, 2006 (gmt 0)

frakilk

A GG, Matt and WebmasterWorld conspiracy ... I love it!

Good things always come in three's.

All the best

Col ;-)

bobmark




msg:734033
 4:05 am on Mar 10, 2006 (gmt 0)

I dunno if this is some sign that Google is scrambling trying to fix something but a lot of dc's worldwide have now reverted to some weird sort of mix of a little bit of the February index with 90% from as old as last August.

However PR has also reverted from what it has been since mid-February to what it was since Sept or so. So have links.

I think Google screwed up bigtime in trying to implement BD. The alternative is they knew it would be this chaotic and damaging and simply didn't care. After all they have always refused to acknowledge their entire business is based on taking the results of the labour of webmasters and using it for their own profit and have always treated everyone but the expedia's of the world as total garbage, so what's new?

I bought a bad bag of bagels from a multi-national for 3 bucks and called the company toll free. Good luck ever getting servive from Google.

My principal hope is they keep on with the M&A so anti-trust will get 'em. They were bad enough B4 - since going public they're even worse.

Grinler




msg:734034
 4:12 am on Mar 10, 2006 (gmt 0)


1. All live pages of your site excluding your home page have been dropped in the BD datacenters.

Exactly.


2. When checking site:mysite.com in a BD datacenter, your home page is the first listing followed by supplemental listings that aren't even live pages in your site? Or once were live listings? Some of the supps I have no idea how google would even find without scouring the root of the public directory in my web server!

Exactly. Now most of my supps do work...but they are old old link formats that I have not used in a very long time so I am not surprised they are there.


3. When checking google.com for a keyword you rank high in, it shows up sometimes, hit refresh and it's gone, however when checking the cache, it appears that it is staying fairly fresh and the cache date is increasing each day?

Not seeing this at all. All my legit high ranking pages are gone from the index entirely though were perfectly visible before the BD update and looks good on non-BD datacenters.

Also have had confirmation that my site is not penalized.

The reality is google screwed up big time and refuse to admit it.

tedster




msg:734035
 4:14 am on Mar 10, 2006 (gmt 0)

Trying to summarize: we have a rapid (but not one shot) change to this condition:

1. Home Page is the only non-Supplemental
2. Supplementals show very old cache dates

I think the appearing/disappearing search results are coming in when the non-BigDaddy data center kicks in for a bit and then gets replaced again. So the Big Daddy supplemental issue really has two characteristics. Except that some people (only a few) have reported the Home Page also going supplemental. That sounds like a different case to me.

The old cache dates are really peculiar, especially with Googlebot spidering so intensely. It's like the recent spidering for some sites never gets into the database, and all that's left is the supplemental.

Questions:

1) What factors make these sites, but not all sites, susceptable to this supplemental issue? Some have suggested that dynamic pages are the ones having trouble -- php driven in particular. So, are there counter-examples to this theory? Static pages (not re-written urls) going supplemental, for instance?

2) Others have suggested that it's the frequently scraped pages that are going supplemental. Does that hold water or are there counter examples? I have no site to look at where many or all pages have been scraped.

3) Another thought was that it's tied to 301 redirects. However, I definitely see counter-examples to that idea. Doesn't hold up for me, at least for a domain-level 301.

mvandemar




msg:734036
 4:24 am on Mar 10, 2006 (gmt 0)

The old cache dates are really peculiar, especially with Googlebot spidering so intensely. It's like the recent spidering for some sites never gets into the database, and all that's left is the supplemental.

I'm actually seeing all the results for one site being nothing but pages that were recently spidered. I know because I added a link to a friends site on all the pages to one of the seo contests going on 2 weeks ago, and now when I search site:domain +newkeyword it's the same results as site:domain. All of the previously cached pages are nowhere to be seen.

-Michael

quarryshark




msg:734037
 4:50 am on Mar 10, 2006 (gmt 0)

"1) What factors make these sites, but not all sites, susceptable to this supplemental issue? Some have suggested that dynamic pages are the ones having trouble -- php driven in particular. So, are there counter-examples to this theory? Static pages (not re-written urls) going supplemental, for instance?

2) Others have suggested that it's the frequently scraped pages that are going supplemental. Does that hold water or are there counter examples? I have no site to look at where many or all pages have been scraped.

3) Another thought was that it's tied to 301 redirects. However, I definitely see counter-examples to that idea. Doesn't hold up for me, at least for a domain-level 301. "

None of the above for me. I do update my pages frequenty, use HTML (manual changes) and have since mid 2000 indexed quite well for my search term...until now.
90% of my URL's have remained the same for the past 2-3 years.
I have a few scrap pages, but not many and I have no 301's.
Oh, and I am being spidered pretty frequently.

bobmark




msg:734038
 5:16 am on Mar 10, 2006 (gmt 0)

I think its a lot more screwed up than that, tedster.

My memory is you were the one pointed out a month or more ago that the BD index was very old - about August vintage. Its like its been interleaved with the Feb index with totally unpredictable results.

I have hadd allinurl results change totally in 48 hours: from the FEB results, to October and now to almost nothing. I see html page supplemental and php pages not as well as the reverse.

I have pages confirmed removed from the index using the removal tool back and supplemental or back and ranking.

I think its so screwed you can't find a tredn other than Google don't care much about if some of us lose money.

mahoogle




msg:734039
 5:25 am on Mar 10, 2006 (gmt 0)

I've seen an 80% reduction in the number of pages listed on a set of data centers via the site:www.missingpages.com. This equates to a 50% traffic loss, probably since the sites full set is still in some data centers.

mvandemar




msg:734040
 5:26 am on Mar 10, 2006 (gmt 0)

Quick almost side-note-but-related... Google counts sidewides in allinanchor counts*. It's one of the things that makes allinanchor slightly unreliable, because they don't appear to be counted the same when determining rankings.

A side effect of this is that if you have good allinanchor in part fueled by a sitewide (or just many links on one domain, doesn't have to be ALL pages), and then a majority of that site gets deindexed (sig links in a forum you are active on is a good example), then your allins will poof as well.

-Michael

*yes, I can show examples if needed.

Grinler




msg:734041
 5:38 am on Mar 10, 2006 (gmt 0)

Another thought was that it's tied to 301 redirects. However, I definitely see counter-examples to that idea. Doesn't hold up for me, at least for a domain-level 301.

I was using zero 301 redirects, am using it now for somet things, and was still affected. Interesting thing for me is almost all the pages I have in supplemental are forum links that utilize 302 redirects.

Abhilash




msg:734042
 5:58 am on Mar 10, 2006 (gmt 0)

Some have suggested that dynamic pages are the ones having trouble -- php driven in particular.

Thanks for the effort, QuarryShark, but the site of mine that was thrown into the Supps has all static pages, and they happen to be in CF as well (don't ask me, i inherited the site). So there's the counter-example to dynamic pages, .php or anything else shifty in nature.

That said, while I appreciate some people (a rare few) who have said their sites have come back already, I regret to report that this site of mine actually got worse. Yesterday at least I had the homepage & sitemap in the index. Now even the homepage is gone, after 5 years of totally legit work online.

GG, please say it ain't so!

Fridaynite




msg:734043
 6:41 am on Mar 10, 2006 (gmt 0)

My supplemental Site is back on the old positions since a few hours and on the most DC.

Maybe, the email helped.

optimist




msg:734044
 7:23 am on Mar 10, 2006 (gmt 0)

--->tedster

To focus on your points in question:

1) What factors make these sites, but not all sites, susceptable to this supplemental issue? Some have suggested that dynamic pages are the ones having trouble -- php driven in particular. So, are there counter-examples to this theory? Static pages (not re-written urls) going supplemental, for instance?

HTML Pages Gone TOO over 200 manually created HTML pages all gone - not even supplimental, only a few supplimentals the rest are no longer in the index. Home Page still showing PR 7 and holding some rankings.

2) Others have suggested that it's the frequently scraped pages that are going supplemental. Does that hold water or are there counter examples? I have no site to look at where many or all pages have been scraped.

Site has been infringed by over 30 companies, most not located in the US, in the past 6 months and gets new cloners and scrapers daily. The deep internal pages, now missing from the index had rare copies made.

3) Another thought was that it's tied to 301 redirects. However, I definitely see counter-examples to that idea. Doesn't hold up for me, at least for a domain-level 301.

Site had 3 pages with 301s. The pages prior to the 301s are showing supplimental, the new pages are nowhere to be found, not even indexed - GONE!
:(

Also found a 302 hijack from a large site with over 3 million pages still in the index. The supplimental originally went to one of the now missing pages. I am trying to remove the hijack now.

inurl:domain.com shows hundreds of proxied searches 75% are 404s the rest are almost all supplimental. Few from my domain.

Some rankings remain the others are hijacked.
:(

Also to note: the non-www is supplimental even though it was 301'd a while back.

sandyeggo




msg:734045
 7:46 am on Mar 10, 2006 (gmt 0)

can I ask a question? Do you guys that are getting hit with big daddy have product related websites or are they information type sites? Is there a trend or is it all various types of content?

asher02




msg:734046
 7:50 am on Mar 10, 2006 (gmt 0)

Does anyone have a website that was effected that DID NOT USE or IS NOT USING 301/302?

tedster




msg:734047
 8:11 am on Mar 10, 2006 (gmt 0)

I just want to focus on the supplemental issue with BD -- that is this thread's title. And yes, there are other crazy issues with BD, but please, so we don't get all scrambled up, let's keep a topical focus in this thread -- supplemental issues with Big Daddy. That's something that is clearly hitting many, many sites.

I understand from a published interview with the Sitemap team, and a comment later on Matt Cutts' blog, that it can take a long time for supplemental pages that really SHOULD be supplemental (once indexed but later 301'd, for instance) to go away. It can take years, and that's not supposed to be a problem.

The problem I am hearing for affected domains is different: urls that are still actively being served are being "eaten away" from the main indexes and into the supplemental index -- until there's no url left on a site: search that ISN'T supplemental but the Home Page. As supplemental, these pages will rarely rank for anything except oddball searches -- even though they were ranking beautifully in the very recent past.

I am sure that Google DOES care about this buggy behavior. Googleguy's post back in part two of this topic is strong evidence of that.

Mar 8, 2006 - GoogleGuy
Hey, I wanted to stop by and give an update. I think we found what the immediate issue was. Future indexing should start picking up most affected people's pages again, though it may take a few more days for it to be visible. I'll check in again after the weekend is over to see if most people are seeing pages returning.

Message #175, found here:
[webmasterworld.com...]

[edited by: tedster at 8:26 am (utc) on Mar. 10, 2006]

graeme_p




msg:734048
 8:18 am on Mar 10, 2006 (gmt 0)

I am seeing the same.

Front pages crawled in the last week or two, everything else very stale: dating back to at least last August, a lot may to July, and at least one page over an year.

The page that is over an year old no longer exists.

Everything but the front page shows as supplemental.

Nothing newer than a few months old shows at all.

Google is also clearly ignoring the re-crawling frequency advice in the Googlesitemap - pages that are set to "never" AND that are are orphaned, have been crawled more recently than pages set to "monthly" and heavily (internally) linked to.

It clearly varies across datacenters. I get visitors who have used Google searches that my sites do not appear in when I do them.

What would fit the facts is this:

  1. Google have some difficulty in transferring the current index to BD.[/li]
  2. They therefore transfer an old index (that is archived), and mark every URL in it as supplemental until it is re-crawled.[/li]
  3. Front pages are either exempted form being marked supplemental, or are re-crawled first, or both.[/li]
  4. Googlebot has to re-crawl everything before things go back to normal[/li]

The interesting question for me is WHY Google have to revert to an old index.

graeme_p




msg:734049
 8:23 am on Mar 10, 2006 (gmt 0)

Sorry, correction to my last post. Pages that are not included in the sitemap have fresher versions indexed than pages that are not in the site map and not linked to from anywhere.

Wibfision




msg:734050
 8:52 am on Mar 10, 2006 (gmt 0)

My site has been affected by the BD supplemental issue as defined by Tedster.

The site is hand coded, pages have the extension .shtml, a 301 redirect is in place diverting from domain.com to www.domain.com. There are also a handful of other 301's in place for other pages within the site. The site is frequently updated and more than half of it has been rewritten in the last 6 months. Duplicate content is not an issue, as far as I am aware, although "snippets" from my site appear in Scrapers now and again.

I, like Tedster, have faith that Google will correct this problem swiftly, although no pages have as yet reappeared on the BD data centres.

RichTC




msg:734051
 10:07 am on Mar 10, 2006 (gmt 0)

What i dont understand is why some authority sites have been nuked by pushing pages into the supplemental index yet some have managed to bypass it!

The one site of ours that got hit badly by this was overnight.

Its a big site with a lot of pages many PR6 and PR5 one moment it had pages in the index, next it was down to its index page and a few loose non relevent pages.

Its like they flicked the switch and wham everything went supplemental and traffic fell off the side of a cliff!

Meanwhile some real cr@p is finding its way to the top following this withdrawal.

The other thing that i dont understand is that even with the pages suplemental, some of them (a few) for whatever reason are showing in the serps in lower positions but most have gone

Currently Googles a mess and this is supplemental hell

Jakpot




msg:734052
 11:11 am on Mar 10, 2006 (gmt 0)

Currently Googles a mess and this is supplemental hell

And consistently inconsistent

asher02




msg:734053
 11:58 am on Mar 10, 2006 (gmt 0)

People need to understand that the supplemental index and the "real" index are totally different entities. meaning that you can mess up the "real" index and these modification will not change the supplemental index. The problem is that we now see only the supplemental index and not the "real" index for the site effected.

I feel that it has something to do with 302/301 as if Google was trying to solve the 302 problem by excluding pages that use 302 but somehow pages that use 301 got also excluded.

The only missing thing for me is that I don't know if there are any other sites that were effected that did not use 301/302 (past/present)

quarryshark




msg:734054
 12:13 pm on Mar 10, 2006 (gmt 0)

"I feel that it has something to do with 302/301 as if Google was trying to solve the 302 problem by excluding pages that use 302 but somehow pages that use 301 got also excluded. "

Well if they are trying to solve a 302 problem, they are failing horribly. My sector is currently stocked with dead domains directing to ebay with more being added daily.

FujitsuBoy




msg:734055
 12:24 pm on Mar 10, 2006 (gmt 0)

I had several pages from a 10,000 page site dip into supplemental hell yesterday, I feared the worse but thank goodness everything seems to be back to normal this morning.

We're regularly using 301s, in fact we just launched a brand new site and I must admit the timing was awful!

Our SERPs seem to have dropped significantly, but because so many changes have happened in the last 2 weeks (both our site and Google), I'm finding it difficult to track down the causes.

RichTC




msg:734056
 1:08 pm on Mar 10, 2006 (gmt 0)

If google itself lost its traffic overnight im sure it would be acting a lot quicker!

Meanwhile whats the bl@@dy point? - have the serps improved - NO, junk is rising to the top and having a field day as a result

Are the serps more relevent - NO junk ranking well as a result of this major blunder

Did it need fixing? - NO - should have left it well alone in the first dam place!

Whats this achieved - Nothing but worse serps and the stuffing of a number of quality authority sites that have supported google pooring millions into adwords!

Do we deserve better than this? - YES

This 210 message thread spans 7 pages: < < 210 ( 1 [2] 3 4 5 6 7 > >
Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google SEO News and Discussion
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved