homepage Welcome to WebmasterWorld Guest from 54.161.155.142
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Google / Google SEO News and Discussion
Forum Library, Charter, Moderators: Robert Charlton & aakk9999 & brotherhood of lan & goodroi

Google SEO News and Discussion Forum

This 213 message thread spans 8 pages: < < 213 ( 1 2 3 [4] 5 6 7 8 > >     
Pages Dropping Out of Big Daddy Index
Part 3
ClintFC




msg:704338
 12:36 am on May 17, 2006 (gmt 0)

< continued from [webmasterworld.com...] >

Seems to me that Matt's recent message confirms my theory. We're either all a bunch of moaning idiots with low quality sites with a few innapropriate, spammy links scattered here and there...or...

The more I think about it the more convinced I am that the missing pages problem is being caused by a Backlink/PR issue (see Msg #15).
Tying together all of the evidence from my own experience, and that of others gleaned from the forums, erroneous or out-of-date backlinks would explain all of the missing pages.

The erroneous, or simply out-of-date, backlink information (which we cannot see) leads to insufficient PR (which we cannot see) and hence deep pages are not indexed.

We all know that a "link:www.mysite.com" does not show you the complete picture. But, since Big Daddy, it now shows just a tiny proportion of backlinks. Way less, than it used to show before Big Daddy. Why? Because either the backlink index hasn't been updated (and now dates back to mid 2005), or else because it has been updated, but the update process is buggy. Only a small handful of Google employees know which of these two possibilities is the case.

We know that the missing pages problem cannot be due to any kind of duplicate content filter, as some people are suggesting. If this were the case, then effected sites would see a proportion of their pages disapear. Some would lose 10%, some would lose 40%, and some would lose 95%. But that's not what we see. We see sites losing the vast majority of their pages or else losing no pages at all. The reason effected sites lose such high percentages of their pages is because of the hierarchical nature of a site. The number of pages increases with depth, and the artificially low PRs (based on innacurate and/or out-of-date backlink data) prevents the deeper content from being indexed.

The fact that Big Daddy was kick-started from an index dating back to the middle of last year, not only explains why the backlink data might be stale, but it also explains why ancient pages keep popping up on various data centres.

As further evidence: try a "link:www.mysite.com" and compare it to a search for "www.mysite.com". In my case, the "link:" search shows just 6 results, only one of which is external to my site. The one external backlink probably pre-dates when Big Daddy's index was seeded. The "www.mysite.com" search, on the other hand, finds hundreds of results representing hundreds of internal and external backlinks. Why aren't these showing up in the "link:" search? Is it because "link:" searches are well known for not showing you the complete picture? Or, has that well-known fact simply been obscuring the true cause of all of the problems? Namely, that the backlinks are simply missing from Google's backlink index.

[edited by: tedster at 8:25 pm (utc) on May 17, 2006]

 

1984bb




msg:704428
 11:30 am on May 18, 2006 (gmt 0)

I have delete all my liks pages...and I don't care anymore about link exchanges ,I have one way links from good quality pages, as I have posted on a new topic that is on hold after the new PRBL update big things are goin to be just remember that post...Just think how many webmasters they will delete whole directories of the usual "links resources" how many of us already stop answer link request emails...I tell you folks after the new PR BL update nothing but nothing will be the same at Google SERPS .I bet my neck that folks that rank at the tops NOW will create tones of threads at WW with the usual moaning and cursing Google when they have lost there top positions due to exploiting PR using heavy link exchange schemes.MCuts told so and it looks like thats all folks...

1984bb




msg:704429
 11:36 am on May 18, 2006 (gmt 0)

Also ...
Q:why Google has downgrade Forums?or Guest books .......?
A:the best doorway to link spam.
good job Google in fact delete all forums that permit html and add link ;).

tigger




msg:704430
 11:39 am on May 18, 2006 (gmt 0)

I've been deleting more link emails than reciprocating anymore - as I said I've become extremely fussy about my linking where in the past if it had PR and looked on theme I would link - not anymore

1984bb




msg:704431
 11:58 am on May 18, 2006 (gmt 0)

Tigger
"I would link - not anymore "
That adds points to my theory that the new Google index after the new PRBL update will be something absolutely new.I will appreciate any comments on that matter.

tigger




msg:704432
 12:08 pm on May 18, 2006 (gmt 0)

I think we are going to see a big drop in PR, at least toolbar PR

RichTC




msg:704433
 12:17 pm on May 18, 2006 (gmt 0)

Anyone got any tips for getting this googlebot to index new pages?

Ive got a PR7 site with five New pages linked from the home page. Each page has about 50 new pages on them yet to be crawled.

The new googlebot has not cashed any of the pages off these five pages and its been this way for weeks now. The five pages are in Google, the pages off them are not.

Frankly i find this new infastructure of googles a pile of junk.

tigger




msg:704434
 12:22 pm on May 18, 2006 (gmt 0)

my PR is not that strong its only a PR4 but I put some new pages up linking from my index and they took 4 weeks to get crawled and the was despite having 20 odd deep links to the new page as well from friends themed sites

ClintFC




msg:704435
 1:51 pm on May 18, 2006 (gmt 0)

The new googlebot has not cashed any of the pages off these five pages and its been this way for weeks now. The five pages are in Google, the pages off them are not.

That's the Big Daddy missing pages problem in a nutshell. Under the new regime (buggy or deliberate - who can say), Google will not index the deeper pages. You need a gizzilian inbound links of exactly the right type (only professional spammers know what these are) in order to qualify for inclusion in Google's index.

This state of affairs will continue until the right person at Google realises what a crock this is...could be a year or two. In the meantime, anyone out there planning on launching an Internet search engine? There's now a gaping hole in the market.

randle




msg:704436
 2:13 pm on May 18, 2006 (gmt 0)

We have a site that started out at 10,000 pages, down to 48 today. However, another site has risen in number of pages indexed. The one that’s dying a slow death has a hyphen in the domain; the one living strong does not. The bulk of the pages on both are from discussion boards.

Anyone else having problems with hyphenated domains? I know Matt mentioned a problem with them in his recent post.

Web_speed




msg:704437
 2:30 pm on May 18, 2006 (gmt 0)

Anyone else having problems with hyphenated domains? I know Matt mentioned a problem with them in his recent post.

LOL, I did see that Matt mentioned that. I too assume that there is a problem with hyphenated domains. Might as well dont you think? it can translate into a couple million more new adwords accounts.

What a joke!

I wounder what’s next on Matt's list? All domains ending with ".com"?

Organizing the world’s information the clueless arrogant way. The new Google$ way.

tiori




msg:704438
 2:38 pm on May 18, 2006 (gmt 0)

Anyone else having problems with hyphenated domains? I know Matt mentioned a problem with them in his recent post.

Yes Matt mentioned it and said he thought it had been fixed. Not sure what he means by "fixed". Maybe he means they've been deleted!

Dayo_UK




msg:704439
 2:45 pm on May 18, 2006 (gmt 0)

I think Matt said it will be fixed this week.

Also - it was to do with site:domain.com (or site:widget-domain.com) not showing supplementals - whether there is a deeper issue I dont know.

To me all these problems could go back to the current calculation of PR which we know Toolbar PR is screwy, old, not updated at the same time for all sites and generally unreliable.

bumpaw




msg:704440
 2:56 pm on May 18, 2006 (gmt 0)

I think it's a sign of how bad things are when you see folks that seldom post whipped into a frenzy. I've noticed over the past couple of months lots over folks with under 50 posts contributing and some appear to be experienced.

mattg3




msg:704441
 2:57 pm on May 18, 2006 (gmt 0)

The algo is now very easy: if search term in wikipedia show wikipedia, if not randomise .. ;)

Anyway now one of my sites has doubled their pages to what it was pre BD, the other is is still at 10% ... traffic is funnily enough not affected. Probably because they now have phantom supplementals.. who knows.

[edited by: mattg3 at 3:17 pm (utc) on May 18, 2006]

vanessa19




msg:704442
 3:00 pm on May 18, 2006 (gmt 0)

Anyone else having problems with hyphenated domains? I know Matt mentioned a problem with them in his recent post.

Our site, which is a hyphenated domain has gone today from .com and co.uk where it has had no1 postion for 3-4 years and page 2 on .com. The company name and domain name are also our search words.

mattg3




msg:704443
 3:04 pm on May 18, 2006 (gmt 0)

Anyone else having problems with hyphenated domains

Our main competitor does not seem to be affected.

we are www.widget.de he is www.widget-xy.de maybe it's different in the European jungle were the weasels live.

Pico_Train




msg:704444
 3:05 pm on May 18, 2006 (gmt 0)

I agree, wikipedia knows all, googipedia

dangerman




msg:704445
 3:11 pm on May 18, 2006 (gmt 0)

Anyone else having problems with hyphenated domains? I know Matt mentioned a problem with them in his recent post.

My experience suggests it may be yet another dead-end, as G throws down sand to cover its tracks. I have two PR5 .com sites, one hyphenated, one not. The hyphenated site is unaffected in the SERPs. The other, my main site and formerly a much heavier trafficked site, is dead in the water as of end of April. But who knows, the way BD is screwing with everything in seemingly arbitrary manner, I am just waiting now for my first site to get squashed.

It is only a matter of time as this site is also useful, well-coded and full of content.

Doesn’t it say something about the craptastic search boffins over at Gurgle that webmasters everywhere, having made sure they dutifully follow the Almighty G’s technical & web guidelines, and act on Spinmeister Matt’s every last word, now need to examine innocuous variables like hyphenation etc etc out of sheer desperation?

King of all Sales




msg:704446
 3:28 pm on May 18, 2006 (gmt 0)

According to Forbes, Google now owns just 43% of total searches and Y and MSN together own over 41%. If you subtract the large number of Google image searches, which are by-and-large useless to most of us, it certainly looks like the tide is shifting.

tigger




msg:704447
 3:38 pm on May 18, 2006 (gmt 0)

>According to Forbes, Google now owns just 43%

not according to my stats its more like 50 / 60% and thats across a good newtwork of sites

gendude




msg:704448
 3:42 pm on May 18, 2006 (gmt 0)

>> According to Forbes, Google now owns just 43% of total searches and Y and MSN together own over 41%. If you subtract the large number of Google image searches, which are by-and-large useless to most of us, it certainly looks like the tide is shifting.

I'm getting more traffic from both of those ratio wise (of course that could be due to Google's abomination of a index update dropping the amount of traffic that G sends my way).

Everyday this week has been worse in regards to indexing. I don't know what's going on, but I play by the rules, work on quality links, quality content, etc., and I watch my stats drop and drop and drop, while all of the spammers going after my niche are doing just fine and don't appear to be impacted.

The good news for me, I've recently had several stories linked by popular/highly ranked sites, which has almost made up the difference in traffic before and after the past week or so.

gendude




msg:704449
 3:54 pm on May 18, 2006 (gmt 0)

It's as if I'm back to the number of backlinks I had around October or November of last year.

Since then I've picked up many backlinks that were considered quality - not spammy, but from high-ranking, very popular sites, and that were natural (as in they posted links to stories on my site that they found interesting).

A few weeks ago I had tons of backlinks that included the ones I've acquired since late last year.

Now none of those backlinks exist. I can goto the sites, and the links are still there, but Google can't/won't see them.

Frustrating as hell.

dangerman




msg:704450
 3:57 pm on May 18, 2006 (gmt 0)

>> According to Forbes, Google now owns just 43% of total searches and Y and MSN together own over 41%. If you subtract the large number of Google image searches, which are by-and-large useless to most of us, it certainly looks like the tide is shifting.

If only... according to the latest statistics from ComScore Networks (March 31) Google's market share rose to 42.3 percent in February, up from 36.3 percent a year earlier. Yahoo dropped 27.6 percent from 31.1 percent, while Microsoft's MSN fell to 13.5 percent from 16.3. These are US-only figures.

It may take several months or longer to see any sustained market reaction to this deterioration in quality of search results.

I have changed our default search away from Google at work, at home and at client sites. Hey, it's a start.

Web_speed




msg:704451
 4:00 pm on May 18, 2006 (gmt 0)

and don’t forget Ask. I've been getting loads of traffic from them recently in addition to good yahoo + MSN traffic.

Google, none (7 years old domains with loads of niche content - go figure, who cares). Thinking of removing the spybar soon. PR is pointless anyway, the main reason why i kept it installed up to now. No need for it anymore, Yahoo's search bar doe's a much better job. The rest is all history as far as I’m concerned time to move on.

Dayo_UK




msg:704452
 4:07 pm on May 18, 2006 (gmt 0)

>>>If only... according to the latest statistics from ComScore Networks (March 31) Google's market share rose to 42.3 percent in February

Yep, and for us in UK and some other Europe countries (Germany I think is very much G orientated) Google market share is 80%+

cleanairguru




msg:704453
 4:14 pm on May 18, 2006 (gmt 0)

I had 240 pages indexed by around 90 of them were suppl. results that were dead doorway pages created by a scum bag SEO compnay in Las Vegas. I fired them and started doing my own white hat SEO and have had good results, but I hated seeing those supplemental results pages. Here's how I got rid of thine within 6 hours.

I uploaded a page that said. "this page has moved, click below ther the new page. I'd put the keyword of hte page as an anchor for the most relevant page on my site. A number of these dead pages are now indexed and helping the website since they have all links to all of our key pages. I have only noticed those suppl. pages dropped as a result and a few new pages added recently.

Nawaralsaadi




msg:704454
 6:14 pm on May 18, 2006 (gmt 0)

Guys, I believe the issue is simple, if we are all losing pages, this means google is losing content, and if google is losing content, this means users are not finding what they are looking for, or at least they are not finding the same depth and choice when searching for a given term, thus if this problem persist users will eventually move on to another search engine should it provide them with better results, the shift wont happen over night, but it will happen, the google guys are aware of this and I believe they will eventually fix the issue or risk losing their king status.

tsm26




msg:704455
 6:30 pm on May 18, 2006 (gmt 0)

Overnight the number of pages we had indexed dropped from 57k to 792. And the pages that google did index is makes no sense. They indexed an entire section on the same level as most of the rest yet dropped everything else. The content dropped is extremely popular and can't be found elsewhere. Page rank and number of backlinks are the same today as yesterday. How this happened overnight is beyond me. Traffic plummeted. Google (worthless) sitemaps checks out okay, no canoninical problem, no duplicate content problem.... MSN and yahoo are looking more relevant every day.

fred9989




msg:704456
 6:40 pm on May 18, 2006 (gmt 0)

Surely the problem here in trying to draw any conclusions about what's going on is the in-out nature of the search results...i.e. your site's in them one day, out the next. (Well, yes, I do mean my site...!)
Are Google toying with us, for fun? Hardly likely. So do they have a purpose?
How about this: they are tracking what actually happens when they present different search results, to see what people who use Google to find stuff actually want. And, on the basis of their conclusions, at some point, they will switch on one or other of the current algorithms - and meet the expectations of the greater proportion of users.
Some of us will be destroyed in the process, Google will not be fulfilling their mission to index (but is that the same as "present"?) the world's information, but they will be serving the majority of their users. Did McDonald's get rich by catering for vegetarians?
Rod

LuckyGuy




msg:704457
 7:10 pm on May 18, 2006 (gmt 0)

fred,

google knows very good what people are looking for, cause they have millions of search strings every day and the tracking. They don´t have to find it out but now.

dudester




msg:704458
 7:30 pm on May 18, 2006 (gmt 0)

just a thought about all of us that will be financially ruined. nope. it takes a few days to rebuild a site on a new domain. if you have decent ideas you can survive no matter what.

about pages dropped. google, unless they are going private again, is gonna have to allow some percieved "junk" in, since it carries adsense. i'd say they will keep some good stuff with good click-through rates, and some junk stuff with fantastic click through rates. adsense ads in my opinion are the only motivator for any flux in rankings. e-commerce and other sites are just affected temporarily until google tries to make sure that adsense carrying sites with not completely junky content are presented first in search results, mix of authority sites with ads and without.

also, if they don't keep listing more pages with adsense, they will stop growing revenues, and shareholders will just appoint a new board of directors that will tell them to index all pages, period.

they will hopefully just present the most relevant results first, followed by crap. backlinks and all that crap are just tactics to get people to talk about things, while g is dancing.

so, sit tight.

This 213 message thread spans 8 pages: < < 213 ( 1 2 3 [4] 5 6 7 8 > >
Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google SEO News and Discussion
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved