| This 213 message thread spans 8 pages: < < 213 ( 1 2 3 4  6 7 8 ) > > || |
|Pages Dropping Out of Big Daddy Index|
< continued from [webmasterworld.com...] >
Seems to me that Matt's recent message confirms my theory. We're either all a bunch of moaning idiots with low quality sites with a few innapropriate, spammy links scattered here and there...or...
|The more I think about it the more convinced I am that the missing pages problem is being caused by a Backlink/PR issue (see Msg #15). |
Tying together all of the evidence from my own experience, and that of others gleaned from the forums, erroneous or out-of-date backlinks would explain all of the missing pages.
The erroneous, or simply out-of-date, backlink information (which we cannot see) leads to insufficient PR (which we cannot see) and hence deep pages are not indexed.
We all know that a "link:www.mysite.com" does not show you the complete picture. But, since Big Daddy, it now shows just a tiny proportion of backlinks. Way less, than it used to show before Big Daddy. Why? Because either the backlink index hasn't been updated (and now dates back to mid 2005), or else because it has been updated, but the update process is buggy. Only a small handful of Google employees know which of these two possibilities is the case.
We know that the missing pages problem cannot be due to any kind of duplicate content filter, as some people are suggesting. If this were the case, then effected sites would see a proportion of their pages disapear. Some would lose 10%, some would lose 40%, and some would lose 95%. But that's not what we see. We see sites losing the vast majority of their pages or else losing no pages at all. The reason effected sites lose such high percentages of their pages is because of the hierarchical nature of a site. The number of pages increases with depth, and the artificially low PRs (based on innacurate and/or out-of-date backlink data) prevents the deeper content from being indexed.
The fact that Big Daddy was kick-started from an index dating back to the middle of last year, not only explains why the backlink data might be stale, but it also explains why ancient pages keep popping up on various data centres.
As further evidence: try a "link:www.mysite.com" and compare it to a search for "www.mysite.com". In my case, the "link:" search shows just 6 results, only one of which is external to my site. The one external backlink probably pre-dates when Big Daddy's index was seeded. The "www.mysite.com" search, on the other hand, finds hundreds of results representing hundreds of internal and external backlinks. Why aren't these showing up in the "link:" search? Is it because "link:" searches are well known for not showing you the complete picture? Or, has that well-known fact simply been obscuring the true cause of all of the problems? Namely, that the backlinks are simply missing from Google's backlink index.
[edited by: tedster at 8:25 pm (utc) on May 17, 2006]
just a thought about all of us that will be financially ruined. nope. it takes a few days to rebuild a site on a new domain. if you have decent ideas you can survive no matter what.
about pages dropped. google, unless they are going private again, is gonna have to allow some percieved "junk" in, since it carries adsense. i'd say they will keep some good stuff with good click-through rates, and some junk stuff with fantastic click through rates. adsense ads in my opinion are the only motivator for any flux in rankings. e-commerce and other sites are just affected temporarily until google tries to make sure that adsense carrying sites with not completely junky content are presented first in search results, mix of authority sites with ads and without.
also, if they don't keep listing more pages with adsense, they will stop growing revenues, and shareholders will just appoint a new board of directors that will tell them to index all pages, period.
they will hopefully just present the most relevant results first, followed by crap. backlinks and all that crap are just tactics to get people to talk about things, while g is dancing.
so, sit tight.
Seems like Matt Cutts team is repairing Google. Some of my pages are reappearing on the following datacenters :
Is there any difference with singular or plural search terms. I have noticed that the words are highlighted as plural yet our site has now dropped for the plural search term but is still listed for the singular although it has a low search term
Why don't most of you believe that Matt Cutts is just giving us spin? One of my sites is down to the home page only. One that is mostly scholarly content lost 50% of the pages. One that is a links based web site with minimal content remains fully indexed.
One of my former competitors, who is no longer in business and no longer has a web site, ranks 5th on first page of search with key words. The site is Cached from last year.
|Yep, and for us in UK and some other Europe countries (Germany I think is very much G orientated) Google market share is 80%+ |
German website: 96% G
English website: Yahoo MSN and G equal.
sample size around 12000 uniques a day.
Complained yesterday about my site going from 300+ to 50+ pages in the last 3 days.
I checked my weblogs and see that my overall hits are up from Google?
This made no sense, so I check individual search terms, and saw that I was coming up with terms I never used to rank for.
I then did a 'site:www.competitor.com' search on my closest competitors websites. They are gone! My closest competitor with a 5+ year old site has 40 pages left out of over 1000. ALL of the smaller mom and pop e-commercers in my field are de-indexed worse than I am!
Why am I not consoled?
I must have missed it and I am feeling rather dense here but where did Matt mention the problem with hyphenated domains and that they are aware?
My site is definitely suffering and we do have a hyphen in the domain name. We dropped overnight from 19k+ pages to just under 600 pages. Probably will lose the last 600 tonight but I am curious if there is any relation.
If you can direct me to where this topic was discussed, I would appreciate it.
|- The team refreshing our supplemental results checked out feedback, and on May 5th they discovered that a “site:” query didn’t return supplemental results. I think that they had a fix out for that the same day. Later, they noticed that a difference in the parser meant that site: queries didn’t work with hyphenated domains. I believe they got a quick fix out soon afterwards, with a full fix for site: queries on hyphenated domains in supplemental results expected this week. |
never mind.. found it.
Oops, thanks mattg3
Looks like we are just being de-indexed because we are small. *sigh* Once Google was small... They have already forgotten what it's like...
>> Frederic 1
I could not find any more pages than the one I see on Google now
Our site is loosing more and more pages everyday
I removed site maps from Google Maps ... nothing seems to happen
Is there anyone at Google reading those threads?
my site is entirely back in the index now. All I had to do was duplicate all my content into a static format instead of dynamic. Now I have duplicates of every page..one static and one dynamic..but all are listed now.
sorry it was a false alert.
finally, my site is still deindexing on those datacenters as well as the others :'(
Friends, don't worry over Google dropping pages. I was anxious too when I saw about 20% of my pages were lost, that was only until I stopped by DMoz and did a couple of tests.
Lets start with dmoz.org homepage. I clicked "Video Games" link. That page is indexed. From there I went to "Action", indexed. Then moved onto "Browser Based", indexed as well. Then to "Asteroids". Aha (as MC says), that page isn't indexed, but has a PR3.
Now repeating the exercise -
Home>Arts>Animation>Anime>Fandom>Fan Works>Fan Art>Web Rings (Not Indexed, PR3)
Home>Science>Social Sciences>Geography>Cartography>Services (Not Indexed, PR5)
Home>Shopping>Vehicles>Autos>Custom and Collector Cars>Customized Cars>Rod and Custom (Not Indexed, PR3)
Do we have sufficient instances to convince ourselves? Pick any category that has many sub-levels, the deeper categories are not indexed.
Now, will Googl risk loosing pages from sites such as Dmoz? I wouldn't think so. If we take what Matt says as right (about sites not having enough link popularity of merit, link exchanges, low quality outbounds etc as reasons behind all these), it somehow doesn't corroborate with tests done on quality sites, and I would tend to think this is an issue that is fixed sooner than later. Relax guys :-)
Some bigger number of indexed pages are back :).
"Friends, don't worry over Google dropping pages. I was anxious too when I saw about 20% of my pages were lost, that was only until I stopped by DMoz and did a couple of tests. "
Did you try those tests at Google Directory :)
just some random I did and of cource no cache
Regional > Europe > United Kingdom > Recreation and Sports > Sports > Country Sports > Associations
Regional > Europe > United Kingdom > Recreation and Sports > Sports > Country Sports > Web Rings
Regional > Europe > Ukraine > Provinces > Khmelnytskyi Oblast > Letychiv
Hmmm...looks like Mcmohan is right try your selfs as well looks like Google deindexing Google :)
The whole thing looks more like "Google Self Destruct" to me.
Last one out of the index turn off the lights please.
1984bb, Google deindexing G directory pages maybe some how correlated to dmoz getting deindexed, am not sure though.
Repeated this one on Yahoo directory. But it is doing better than dmoz. However the paginated pages (after the 1st page) are not indexed in many instances I checked. Not enough trusted links perhaps to Yahoo :-)
I have never posted in this category before so I am a little nervous.
Some time ago (early May) GGuy gave a title code that we could use to contact the search team to report dropped pages (Cannot remember code word)
I wrote (expecting the usual cut-paste reply) However yesterday I had an email back saying that it is likely that the dropped pages may be back within the week. Something about datacenter fluctuations.
It is all double-dutch to me but it would be nice to see Google get back to a reliable index. I tend to use other engines if I am really looking for something.
>Some time ago (early May) GGuy gave a title code that we could use to contact the search team to report dropped pages (Cannot remember code word)
try "crawlpages" within the title
Good for you. They must have spotted something there. At least you got a response.
sorry Sense_able I miss understood your posting - need more coffee, well done on getting a reply
|may be back within the week |
mmmm MAY come back and WHICH week, this ones done and dusted already, although most movements do happen over weekends I have noticed Mondays show changes, the first view we all got of the new SERP,s was 2 Mondays ago
"sorry Sense_able I miss understood your posting - need more coffee,..."
Coffee wouldn't do it. Try Cappuccino ;-)
Since two weeks my site pages to dropped out of index and showing only the home page in Index.
I have submitted my sitemap to Google 2 days ago. Today I can able to see the report generated and it reads "No pages from your site are currently included in Google's index. Indexing can take time." and it also says that "Googlebot last successfully accessed your home page on May 16" i.e. the day which I have submitted my sitemap.
Can anyone here predict how long might take to index pages after submitting sitemap.
I found the sitemap offered no benefits at all, so long as you have your own thats good enough IMHO
reseler"Coffee wouldn't do it. Try Cappuccino ;-) "
I unfortunatelly can't drink anymore coffeine coffee because some time ago Google send me to the Hospital with Heart palpitations , since then the doctors advice me not to Smoke ,Drink ,no coffeine ,no stress with Google ,I rememper the date it was just before xmas 2005 when one night I sudenly show my site's top possitions disappearing and go at page 80.....
Since then my life has change a lot ,after a period of deep depression and nearly on edge of divorce because I had become an old grumpy old man and all just because of Google ,i have invested all my last 3 years internet revenues on building an existense on something that is controled only by me (thanks to CJ I made a small hotel in a small Greek island)
|"Friends, don't worry over Google dropping pages. I was anxious too when I saw about 20% of my pages were lost, that was only until I stopped by DMoz and did a couple of tests. " |
There are a vast amount of high level Dmoz pages not cached by Google when I last checked. You would assume therefore that dropped pages has something to do with a sites content rather than the inbound links. Dmoz must have some of the best incoming links on the web and still it has this problem. Obviously it is doing something Google doesn't like presently.
I am generally not a believer in conspiracy theories, but has anyone stopped to consider that this entire BigDaddy fiasco may actually have a motive? It's a very efficient way for Google to test how many webmasters and small businesses will convert into Adwords customers after their sites have been decimated. Google is giving us all a good hard PUSH in that direction by "suddenly" screwing up the index. "Ooops! Sorry your traffic has vanished inexplicably...NOW PAY US." I for one plan to rebuild my site to try an rank better with Yahoo and MSN rather than pay Google to advertise. The most sensible thing we can all to is try to become less dependent on any one SE for our business.
|I am generally not a believer in conspiracy theories, but has anyone stopped to consider that this entire BigDaddy fiasco may actually have a motive? It's a very efficient way for Google to test how many webmasters and small businesses will convert into Adwords customers after their sites have been decimated. |
Spot on! i just don't buy the crap and smoke screen various google reps are trying to build. They are openly saying that they are going to look closely into sites with affiliate links and incoming outgoing site links....opps there goes 95% of the web....how convenient.
Pathetic really!, i just hope those affected will not fall for it and (as a matter of principal) will use the Yahoo & MSN ad networks. Not Adwords.
Any webmaster that has some respect left for his/hers site(s) should immediately stop supporting this octopus. We do have the power to tip the scales if we realy want to. Do your part.
Agree entirely with previous few posts. Is their prime objective to rid the web of affiliate sites? Their money comes from adwords not adsense. Judging by the flotsam and jetsam in search results over the last two months, what a lot of collateral damage. And the worst sites are still up there!
Meanwhile we are picking over the bones of hyphenation, 301 www/non-www, canonization.. phooey! Hate to put a spanner in the works but my $20k/mth adwords budget is moving steadily across to Yahoo and to MSN when AdCenter launches in the UK. Let's get moving!
>> I am generally not a believer in conspiracy theories, but has anyone stopped to consider that this entire BigDaddy fiasco may actually have a motive? It's a very efficient way for Google to test how many webmasters and small businesses will convert into Adwords customers after their sites have been decimated.
It's an interesting theory, and people are searching for explanations about why Google has botched this up so badly.
The problem is, while I'm sure Google wants to get more people into AW, if they make it hard for people to find sites running AS, then they are shooting themselves in the foot.
There's also the anger and trust factor - I have a service that I use AW to promote, and I also have sites with AS on them (the two are seperate I don't actively use AW to promote my AS sites, although sometimes if I'm launching a new site I might run AW for a week or two just to pick up some mindshare and get noticed), and I have two perspectives:
1)As an AW user, I see the problems with my sites, and I have to question whether my AW dollars are being spent wisely - I have the feeling that my Ads will not run on the most relevant sites, and that I am wasting money. If my sites are not being indexed properly, and many others are reporting the same, then I just don't have confidence as an AW user.
2)As an AS publisher, I'm agry that Google has botched this up, and have no desire to keep using AS, and this carries over to AW - why should I give money to a company that screws up like this? Why should I run AS that benefits Google as well, when there are a lot of other advertising programs out there?
It makes no sense why Google has botched this up so badly - it benefits them to index as much as possible, relevant sites that are going to help with AW/AS.
It pisses me off that the spam sites in my niches are doing just fine in the indexes while I've been hammered.
Heck, maybe because the spam sites are spam sites and geared towards getting people to click an ad and leave, rather than keeping people around and informing them like many of us do, that the spam sites are more valuable to Google (or at least to Google's indexes).
I'm not saying that Google (the company and people that make up Google) wants spam sites, I'm saying that Google's software is keeping the spam sites around because they are better for AS.
| This 213 message thread spans 8 pages: < < 213 ( 1 2 3 4  6 7 8 ) > > |