homepage Welcome to WebmasterWorld Guest from 54.204.67.26
register, free tools, login, search, subscribe, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor
Home / Forums Index / Google / Google SEO News and Discussion
Forum Library, Charter, Moderators: Robert Charlton & aakk9999 & brotherhood of lan & goodroi

Google SEO News and Discussion Forum

This 213 message thread spans 8 pages: < < 213 ( 1 [2] 3 4 5 6 7 8 > >     
Pages Dropping Out of Big Daddy Index
Part 3
ClintFC




msg:704338
 12:36 am on May 17, 2006 (gmt 0)

< continued from [webmasterworld.com...] >

Seems to me that Matt's recent message confirms my theory. We're either all a bunch of moaning idiots with low quality sites with a few innapropriate, spammy links scattered here and there...or...

The more I think about it the more convinced I am that the missing pages problem is being caused by a Backlink/PR issue (see Msg #15).
Tying together all of the evidence from my own experience, and that of others gleaned from the forums, erroneous or out-of-date backlinks would explain all of the missing pages.

The erroneous, or simply out-of-date, backlink information (which we cannot see) leads to insufficient PR (which we cannot see) and hence deep pages are not indexed.

We all know that a "link:www.mysite.com" does not show you the complete picture. But, since Big Daddy, it now shows just a tiny proportion of backlinks. Way less, than it used to show before Big Daddy. Why? Because either the backlink index hasn't been updated (and now dates back to mid 2005), or else because it has been updated, but the update process is buggy. Only a small handful of Google employees know which of these two possibilities is the case.

We know that the missing pages problem cannot be due to any kind of duplicate content filter, as some people are suggesting. If this were the case, then effected sites would see a proportion of their pages disapear. Some would lose 10%, some would lose 40%, and some would lose 95%. But that's not what we see. We see sites losing the vast majority of their pages or else losing no pages at all. The reason effected sites lose such high percentages of their pages is because of the hierarchical nature of a site. The number of pages increases with depth, and the artificially low PRs (based on innacurate and/or out-of-date backlink data) prevents the deeper content from being indexed.

The fact that Big Daddy was kick-started from an index dating back to the middle of last year, not only explains why the backlink data might be stale, but it also explains why ancient pages keep popping up on various data centres.

As further evidence: try a "link:www.mysite.com" and compare it to a search for "www.mysite.com". In my case, the "link:" search shows just 6 results, only one of which is external to my site. The one external backlink probably pre-dates when Big Daddy's index was seeded. The "www.mysite.com" search, on the other hand, finds hundreds of results representing hundreds of internal and external backlinks. Why aren't these showing up in the "link:" search? Is it because "link:" searches are well known for not showing you the complete picture? Or, has that well-known fact simply been obscuring the true cause of all of the problems? Namely, that the backlinks are simply missing from Google's backlink index.

[edited by: tedster at 8:25 pm (utc) on May 17, 2006]

 

Hissingsid




msg:704368
 8:00 am on May 17, 2006 (gmt 0)

Hi,

Post Florida we all noted that directories and sites with many outbound links did better than those with none. I added links to authority sites on the same topic as our site on the home page and throughout the site. We have stuck at #1 since. But on some of the Big Daddy DCs we have disappeared.

I wonder if the algo is looking for unusual out link patterns associated with large volume search terms. Perhaps if the terms that your pages used to rank for have enough volume to produce a graph in "Trends" then they are looked at more closely.

If I were trying to improve search quality I would focus on the terms that folks actually search for. No one is going to complain about crap serps for terms that they don't use.

Sid

Soulbeatle




msg:704369
 8:14 am on May 17, 2006 (gmt 0)

Iím getting my head in a bit late but anyway. Our Swedish (geographical adjective) site experienced the dreaded drop a couple of weeks ago. We had about 18000 at site:.domain.com and are at this point smothered to about 148.

The theory about crap in-links could, Iím afraid to say fit in quite accurately on our story. Being all new, with a great and serious idea though, we where referred to swaps and directories and saw a good climb in indexing and visitors. Quality links were also coming in as people discovered the benefits of our site. Now, starting a few steps back again, we are committed to get some quality in-links.

If BD is successful in adding relevance to search results, Iím all for that change. Our idea is built on relevance and we will succeed, but still, there have to be some way to get in the system, being new and wonderful. ;)

tigger




msg:704370
 8:22 am on May 17, 2006 (gmt 0)

I agree Relevancy

but what if lets say you have a book review site

you build a solid site with the bulk being your own content then you theme the books into sections say romantic, sci-fi and so on

then from a well know book affiliate you use snippets ( not the whole review) to preview the books which you then build pages around this say 10 books per page, with each book linking to the affiliate to buy it and within these pages would be your own content

would that in your view be just "another" affiliate site ?

1984bb




msg:704371
 8:41 am on May 17, 2006 (gmt 0)

"My view on the affliate links is that if your site has nothing more then affiliate external links and product dup content then you are no more valuable then any other site "
Are you sure? anyone can sticky me to show a German page that rank at tops for any destination for years now with 90% zanox and other affiliate links at the bigest money travel sector in Germany....

The_Tank




msg:704372
 8:42 am on May 17, 2006 (gmt 0)

Right then - been trying to play catch up - its only taken me the best part of an hour or two to read matts blog entry and this discussion.

I think I must be the unluckiest site owner out there. I don't link to non relevant sites. My site isn't part of a link network. I do have links from some well established relevant sites online. I do have some links from sites which are non relevant - but I don't control them so not much I can do there.

Yet I look at online competitors and they still have cloaking pages, JavaScript redirects and are obviously part of link exchange programs as they link from widget sites to hotels etc.

So I sit here with 90% of my site content still not indexed (slight improvement recently). Its one battle to get it ranked high in the index - but should it be so hard to get it included in the first place? I thought Google wanted to index the 'whole' web.

I haven't had a reply from the gmail address and the way things are going - if I did it would probably be caught by my email spam filter and I wouldn't get to read it.

ClintFC




msg:704373
 9:11 am on May 17, 2006 (gmt 0)

If BD is successful in adding relevance to search results, Iím all for that change. Our idea is built on relevance and we will succeed, but still, there have to be some way to get in the system, being new and wonderful. ;)

According to Google's new way of thinking, you need to have a large number of "natural" inbound links or they won't even index you. Natural links do not include bought links, reciprocal links, links from bad neighbourhoods, or links that they deem to be off-topic for your web site.

I was explaining this to my two year old daughter when she stopped me short and pointed out the fatal flaw:

You need natural links or we won't list you. How do you get natural links? People find your site, like it a lot, and therefore add a link to your site from theirs. How do these people find you in the first place? Well, on a search engine of course...oops!

Gimp




msg:704374
 9:44 am on May 17, 2006 (gmt 0)

Clint - You advertise. That is what any new business does.

ClintFC




msg:704375
 9:51 am on May 17, 2006 (gmt 0)

You mean you buy links? But that's illegal under the new Google laws.

tigger




msg:704376
 10:04 am on May 17, 2006 (gmt 0)

is this not the big debit whats the difference between buying links for links or buying links for traffic!

Gimp




msg:704377
 12:43 pm on May 17, 2006 (gmt 0)

I can't believe it. Advertise means advertise.

There is a world beyond google and there is a world off line. And you should know that can use paid advertising in the search engines.

Join the real world and do some old fashion marketing. That is how people find you.

cbartow




msg:704378
 12:51 pm on May 17, 2006 (gmt 0)

You mean you buy links? But that's illegal under the new Google laws.

So shouldn't I be penalized for buying AdWords?

tigger




msg:704379
 1:01 pm on May 17, 2006 (gmt 0)

its not adwords, or any of the other PPC SE's its more people that buy links from popular sites in the hope it will send traffic thats the issue - are you buying a link or buying traffic

but my concern is the affiliate side of this as "if" I'm reading the blog correct all affiliates sites are going to get hit in some way or another depending on how much content you use from the affiliate - so if you have this book type review site you've had it as you won't be able to use any content

LuckyGuy




msg:704380
 1:57 pm on May 17, 2006 (gmt 0)

It is the dopiest thing that google has done in the last years. We are now all forced to get links from a precise matching niche. Else it wonīt work. How to get it if not a big business with a lot of assests in background?

This was not a brilliant throw to put all sites in one basket. I thought google engineers would be more slier. Instead of looking at the real spammers they wipped out all but the big ones.
Besides I saw a site with 2400 backlinks all from one forum. That forum is driven be themselves. They are #1 for mayor keywords, thats the reality.

But not to forget about the other techhnics. Keyword stuffing works fine, cloacking works fine, javascript redirect works fine. It time to change way of thinking if google does not remain true to themself.

Last but not least has google forgoten to think about the niches where it does not remunerate to participate on adwords or to register at a price comparison site. There is a big leak of thinking. If it goes on like this we donīt need google anymore because there are only some few well known pages in index that you can trigger without google.

Just my opinion,

regards
LG

M3Guy




msg:704381
 3:11 pm on May 17, 2006 (gmt 0)

Looking through the serps in my niche it seems as though all the sites that have dissapeared are in one way or another affiliates of a sort.

But the daft thing is that the big ones that have dropped have got loads of unique content on their pages and have not simply ripped content from the affiliates they are promoting.

It almost seems as though G have added a penalty for sites that have any form of affiliate code in their outbound links ie. affid=? partnerid=? etc.

F_Rose




msg:704382
 3:16 pm on May 17, 2006 (gmt 0)

We have blocked our affiliate links with a javascript:void(0); for the SE not to follow the links.

However, our site was affected as well.

sodascouts




msg:704383
 3:19 pm on May 17, 2006 (gmt 0)

So the new rules are that you should seek out sites related to you in order to get "quality links" - but don't link back to them, regardless of their quality or mutual relevance. Only put links to quality sites if you know Google will think them directly relevant and they have NOT linked to you. If in doubt, remove the link, regardless of whether you think it would be helpful to a user. (Better take off that link to your favorite charity or - heaven forbid - charitIES). Otherwise, Google removes 90% of your pages from its index ENTIRELY. Obviously everything you do is crap if you have "too many" (5? 10? 20?) outbound links Google considers irrelevant, regardless of their quality. Playing their game means abandoning the user in favor of the Googlebot.

Ridiculous... and hypocritical. Why is Google trying so hard to control what webmasters consider valuable? If this is supposed to cut down on spam, why are so many spam sites thriving while small sites get wiped out? So much for letting a new site naturally develop links.

I don't rely on my site to generate income, so I'm not going to bother trying to please Google. Guess I'll stay in supplemental hell forever for my lack of unnatural efforts. Oh well - at least my site is still high ranking on Yahoo and MSN. Perhaps I will put links on my site to them. (GASP! Such irrelevance!)

Google used to be the engine you turned to because it had the most indexed pages and you were bound to find something, even if you had to fish for it. Now it's reversing that and removing indexed pages, and from Matt Cutt's blog, apparently they think this is a terrific idea. Reminds me of the New Coke fiasco in the mid-80s (Coke is the most popular soda - let's change how it tastes!) We all know how that went over.

Perhaps Google believes that no one will notice. This is probably true for generalized terms. But when a user searches for specific content and gets few or no relevant results in Google, they are forced to go elsewhere. It's as simple as that. And that is how the big guys go down - forgetting that no matter how long you've been on top, you have to keep up the quality, or you won't stay there. At least Bill Gates should be happy.

I can only hope this is spin and not really the way they have structured BigDaddy. Surely these high-paid people at least have common sense.

gendude




msg:704384
 4:23 pm on May 17, 2006 (gmt 0)

It's ridiculous.

The spam sites that are in my niche haven't suffered at all, while I've had my results cut way down. I've seen the same thing with a couple of friends - they've been impacted even worse than I have.

It could be that people are stealing my content, and I'm the one getting penalized, and I'll have to look into that this weekend, but that doesn't help me out much - I can work with Google and get them delisted, but I doubt Google will give me any kind of special re-index - my sites only handle 50,000 or so people a month - a drop in the bucket.

In addition to my websites, I have a small side service that I use AdWords to advertis - I'll move that AW money over to another advertising service, and I'll switch out my AdSense ads on my website to other services as well. If Google calls or emails like they have in the past when I've made such changes, I'll let them know, but they are seperate entitites from the Google SE gods.

Google may not notice, but I will feel like I have done something :-)

Just Guessing




msg:704385
 4:26 pm on May 17, 2006 (gmt 0)

why are so many spam sites thriving while small sites get wiped out?

because some spammers know how to acquire "high quality" IBLs.

randle




msg:704386
 4:36 pm on May 17, 2006 (gmt 0)

because some spammers know how to acquire "high quality" IBLs.

Hence the trials and tribulations of an algo thatís heavily based on link popularity.

Relevancy




msg:704387
 4:40 pm on May 17, 2006 (gmt 0)

>> would that in your view be just "another" affiliate site?

Simple.. are you just a store with books or are you a store with books, community forums, book blogs, User hand written reviews of the books, reviews of the best libraries, etc. If not, you better win by having the best natural links from the most powerful sites to offest your dup content/affiliate link limited site.

Relevancy




msg:704388
 4:43 pm on May 17, 2006 (gmt 0)

>> but my concern is the affiliate side of this as "if" I'm reading the blog correct all affiliates sites are going to get hit in some way or another depending on how much content you use from the affiliate - so if you have this book type review site you've had it as you won't be able to use any content

You wont be hit by anything, you will just have less chance of ranking unless you get better links (naturally ofcourse). You dont get penalized you just get looked at as less important.

jrs_66




msg:704389
 4:50 pm on May 17, 2006 (gmt 0)

The blunt core fact of all this mess is that IBL based algorithms are no longer relevant to providing the best SERP's. Google seems to have wed themselves more this this antiquated idea, rather than moving away from it.

Relevancy




msg:704390
 4:51 pm on May 17, 2006 (gmt 0)

I am hoping for all of us that this is just a time and age thing and that just becasue our pages dropped due to the lose of reputation from the crap links, we will still get indexed sometime even without links.

soapystar




msg:704391
 4:52 pm on May 17, 2006 (gmt 0)

because some spammers know how to acquire "high quality" IBLs.

This is very misleading. While the statement is accurate for some of course, the majority of Spammers are thriving because they have aquired networks of sites they can manipulate the links from. These are not quality links, they are just not reciprocal. Over the past year Googles ability to spot networks has weakened considerably.

Relevancy




msg:704392
 5:01 pm on May 17, 2006 (gmt 0)

Quality link building is easy if you have the money. All Google did is delay the little guys again and let large sites/companies own the web. It is funny if google wants more adwords revenue, they are going about it wrong. The little guys dont have the dough to spend.

dangerman




msg:704393
 5:02 pm on May 17, 2006 (gmt 0)

It almost seems as though G have added a penalty for sites that have any form of affiliate code in their outbound links

Well funny you should say that. My site has lost 2000 visitors a day from targeted niche sectors. We have relevant Yahoo (new commercial directive: penalize Yahoo partners?) advertising links on most pages, yet secondary to our main content. I know many in the same boat just as pissed off. It is sickening the way webmasters are forced to hang to every dripping word of Mr Cutts like a pack of starving mongrels.. but where else can they go?

tigger




msg:704394
 5:18 pm on May 17, 2006 (gmt 0)

>I am hoping for all of us that this is just a time and age thing and that just becasue our pages dropped due to the lose of reputation from the crap links, we will still get indexed sometime even without links

I hope your right but I'm not holding my breath

Regarding my book posting

My site has a good mix of my own content mixed in with affiliate information, but as in my book example the content provided by the affiliate still only amounted to 40% of the content the rest was my own and these affiliate pages where 3rd & 4th level pages

I've been some further research across a wide selection of keywords and I'm sure this latest twist of the knife is aimed at affiliate sites as I'm struggling to find any

Relevancy




msg:704395
 5:29 pm on May 17, 2006 (gmt 0)

It is not aimed at anyone other then crap back links and quality ratios. Sites with tons of irrelevant outbound links will be affected just the same as a site with all affiliate outbound links.

I run or am associated with numerous sites.. some affiliate oriented some not.. What seems to hurt the ones that lost pages the most is the fact that backlinks where hit and therefore deep indexing was stopped or slowed. It is just that most affiliate oriented sites tend to be connected one way or another to bad links.

Big Daddy = Quality Control Check

tigger




msg:704396
 5:48 pm on May 17, 2006 (gmt 0)

well the links I have leaving my site and theres not many are only to relevant sites & this applies to other friends I have spoken to that have been hit by BD

longen




msg:704397
 6:33 pm on May 17, 2006 (gmt 0)

I haven't been affected by the dropped pages that some have experienced, until this week.

On the 9th May I did a 301 redirect from page.htm to a another of my sites as follows:
RewriteRule ^page-a\.htm$ [domain.com...] [L,R=permanent]

What i overlooked was that over 600 other pages link to page.htm. I now find that many have gone missing from the Google index. I can't be sure when they dropped out, but i will assume it was over the last week. I have now removed the Redirect to see if they come back.

F_Rose




msg:704398
 6:44 pm on May 17, 2006 (gmt 0)


F Rose Wrote:
"Based on Matt Cutts post..
We have a site that is half affiliate and half our own roducts with our own shopping cart...
Are we being penalized for having affiliate links?
If that's the case what should we do get rid of half of our income due to Google?"

Reseller Wrote:
I guess Matt talks mostly about an affiliate site should have "Value added" content. And not duplicating the content of affiliate program vendors, or just having collection of affiliate links. I.e he is still talking about "Thin Affiliates" .

Another Answer I got:
Don't overreact. The affiliate links Matt used as examples were in the footer and looked like run of site links. He made a point of using terms like "not related." I would think that if your affiliate links are well incorporated and not just scattershot or run of site you *should* be okay."

Based on what you are debating now, was I really overreacting with my post yesterday?

This 213 message thread spans 8 pages: < < 213 ( 1 [2] 3 4 5 6 7 8 > >
Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google SEO News and Discussion
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About
© Webmaster World 1996-2014 all rights reserved