homepage Welcome to WebmasterWorld Guest from 54.205.247.203
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Google / Google SEO News and Discussion
Forum Library, Charter, Moderators: Robert Charlton & aakk9999 & brotherhood of lan & goodroi

Google SEO News and Discussion Forum

This 38 message thread spans 2 pages: 38 ( [1] 2 > >     
Drastic drop in pages indexed - due to Big Daddy?
leoo24




msg:707337
 6:24 pm on Feb 21, 2006 (gmt 0)

hey guys :)

site:domain gives me 130k pages, BD gives me 20k, but using the API always gave me around 10k, but yesterday i suddenly lost 8k pages!

anyone else experienced a huge drop in real indexed pages, as opposed to site:domain?

i read someone comment that his own opinion about sites losing pages was due to BD, from that comment i would assume others have also experienced the same as me.

 

Tearabite




msg:707338
 7:53 pm on Feb 21, 2006 (gmt 0)

I have a small, new site, and BD never had any more than 50% of my pages, non BD had about 75%.
As of today, BD has 100% and non-BD still has the same 75%

Give the Big Daddy time to catch up.. they've got more than a few pages to chew thru..

.

leoo24




msg:707339
 8:53 pm on Feb 21, 2006 (gmt 0)

thanks for the reply.

yeah i'm not entirely bothered by BD but rather the 8k drop via the API.

i have another site with 150k urls in BD but only 2 in others, but teh API also shows only 2.

insight




msg:707340
 11:00 pm on Feb 21, 2006 (gmt 0)

We have seen this as well - doing a "BigDaddy" IP search for "keyword site:www.example.com" gives 630 pages, but "keyword site:www.example.com/subdirectory/" gives 1200. So there are more pages found in just a subdirectory than on the site as a whole

The first search above gives 20000+ pages when searched on a non-"BigDaddy" IP address, and page-by-page spot checks shows less indexing/caching with "BigDaddy."

ctor




msg:707341
 7:21 pm on Feb 22, 2006 (gmt 0)

My site lost every single page in the site: search, but when searching for the main keyword, the site still showed up (on page 2 though, instead of early on page 1).

Yesterday there were 3 pages visible from site:, and today there are 2 pages and 10 links, and the position on page 1 is restored.

Total pages on the site is about 80.

BillyS




msg:707342
 7:45 pm on Feb 22, 2006 (gmt 0)

On BD I show 355 page and about 100 of those are supplemental. On BD, the number of pages has gone from 450 to 890 to 450 to 350.

On non-BD I show 1,020 and the number continues to increase each week as pages are added to this website.

Googlbot is a constant visitor and I'm not the least bit worried about the page difference (other than the fact site search is horrible when it hits a BD DC).

I suspect that at a future date, there will be a massive update of the BD centers where a good amount of these missing pages will appear.

nzmatt




msg:707343
 3:33 am on Feb 23, 2006 (gmt 0)

I have 14,800 for 'site:' on BD, 10,000+ on others.

But when I search the individual urls on BD:

"Sorry, no information is available for the URL..."

Yet the page is still ranking in the serps (BD or otherwise)...?

Steph_R




msg:707344
 3:52 am on Feb 23, 2006 (gmt 0)

leoo24,
Are you using sitemaps?

dinnerware




msg:707345
 4:01 am on Feb 23, 2006 (gmt 0)

Yes, we have lost a ton of pages and the pages that have not been indexed are obviously not ranking. Without question, this is a problem with Big Daddy. So the question.. Is this temporary? Has anyone else heard any theories/explanations? This Big Daddy has really hurt our traffic.

Steph_R




msg:707346
 4:07 am on Feb 23, 2006 (gmt 0)

dinnerware,
Are *you* using sitemaps?

dinnerware




msg:707347
 4:10 am on Feb 23, 2006 (gmt 0)

We have a Site Map but do not use "Google Sitemps"

nzmatt




msg:707348
 5:24 am on Feb 23, 2006 (gmt 0)

"Sorry, no information is available for the URL..."

Yet the page is still ranking in the serps (BD or otherwise)...?

How is this possible?

BTW This site has no G site map yet.

Halfdeck




msg:707349
 7:13 am on Feb 23, 2006 (gmt 0)

The number of pages indexed for one of my domains have been increasing steadily since the beginning of Feb. till a few days ago when number of indexed pages suddenly dropped from 800+ to 240 on many of the DCs. Earlier tonight, the number of pages listed on 216.239.59.147, 64.233.183.104, and 66.102.9.104 rose to around 700+ with 2/22/2006 cache dates while the BDs show older cache dates with either 240 or 1,300 pages indexed, latter containing around 1,200 supplementals.

I'm keeping my fingers crossed.

Nikke




msg:707350
 7:24 am on Feb 23, 2006 (gmt 0)

This is strange.

You all report on fewer pages reported from BigDaddy, but try searching for site:.com on a BD vs. non-BD datacenters.

I see more pages in Bigdaddy. Not that many more, but more. Still, for my own domain I get 100 less pages on BD than on non-BD.

dakman




msg:707351
 9:14 am on Feb 23, 2006 (gmt 0)

google with BD seemed too wipe a big part of the index... which i think they didnt need to do... why couldnt they apply the new algo's to the existing pages they have indexed?

my page counts have dropped 1,000 % since BD's been propagating but finally are steadily increasing... my page counts went from 100k to just 800 pages then back up to 18,000 today.

I've noticed the same about my compeition and countless of other sites...

dakman




msg:707352
 9:16 am on Feb 23, 2006 (gmt 0)

it seems like BD is starting to catch up in index page counts vs the other dc's

it took BD a month after it seemed to have thrown out pages and re-index them again...

leoo24




msg:707353
 4:26 pm on Feb 23, 2006 (gmt 0)

i'm not using sitemaps no, rely on crawling, old fashioned i guess :D

my site that lost many pages in BD and via API is 2 years old, currently 130k pages using site: in non bd dcs, 20k in BD and 2k using API

a new site, maybe 3 months old, is 2 pages only in non BD dcs, 170k pages in BD and 12k pages using API

Thew new site is increasing pages in BD, and until a few days ago showed 0 pages using API. This new site is clearly increasing, whilst the older site is dropping in BD and API.

dinnerware




msg:707354
 7:42 pm on Feb 27, 2006 (gmt 0)

Has anyone seen an improvement of their dropped pages? We have not, we seem to be losing more.
Anyone have insight, encouragement...?

dinnerware




msg:707355
 10:39 pm on Feb 27, 2006 (gmt 0)

No we do not but I have talked to several people who use them and do not and those that do have been dropped also.

powerstar




msg:707356
 10:44 pm on Feb 27, 2006 (gmt 0)

Also losing pages

PR8 and PR7 pages...wired. The pages left are still ranking ok but missing alot of the main pages

leoo24




msg:707357
 11:03 pm on Feb 27, 2006 (gmt 0)

yeah i've had no hit in rankings or visitors at all, just indexed pages

pages have crept up slightly, but every 5 minutes it keeps changing.

rookiecrd1




msg:707358
 2:15 pm on Feb 28, 2006 (gmt 0)

It seems liek I moved up for the pages that are still indexed, but there are also about 150,000 pages missing now which sucks.

Will this pages get reindexed?

dinnerware




msg:707359
 2:54 pm on Feb 28, 2006 (gmt 0)

>>>> Will this pages get reindexed? <<<<

This is the million dollar question....

Wibfision




msg:707360
 4:57 pm on Mar 1, 2006 (gmt 0)

Rookiecrd1 - my site is in the same position as yours - I suspect the increased ranking on the pages that do exist on BD is due to the pages that would normally rank above us also being missing.

dopecoder




msg:707361
 9:28 pm on Mar 1, 2006 (gmt 0)

incidently one of my site dropped from 98k to just 131 on site:www.mysite.com, the trafic doest seem to be affected but i guess its too early to tell.

i was just going throught these posts, can anyone give me a big daddy ip and a non big daddy ip so i can compare the difference.

Armi




msg:707362
 12:04 am on Mar 2, 2006 (gmt 0)

My site dropped dramatically today.

From 2.500.000 (Wednesday 22 February) now to 227.000 :-(

rookiecrd1




msg:707363
 12:41 am on Mar 2, 2006 (gmt 0)

After dropping from 250k to 74k in the last week, it just dropped again to 24k.

I hope this isn't permanant. I am really baffled. We are what I think an authority site, and now only have about 10% of out pages indexed.

Armi




msg:707364
 9:11 am on Mar 2, 2006 (gmt 0)

Now, I have only old supplementals in the index :-(

seochristine




msg:707365
 10:04 am on Mar 2, 2006 (gmt 0)

It seems all the Google Engineers have left for the SES NYC in the midst of BigDaddy.
Many well to do old established sites are seeing the same sort problems, dropped from BD index and showing supplemental results in "site:" command.
Even I have been subjected to the same "searchquake". But to the utmost surprise there hasn't been a sharp decline in traffic, or hits from the robots, nor has there been any change in Pagerank.
It seems Matt and his team are totally unaware of this. The BigDaddy and its predecessors are seriously causing panic for many.
Can someone please tell, how long should I hold my patience? Of course there is no other way out also other than waiting for things to settle permanently.

Dayo_UK




msg:707366
 10:06 am on Mar 2, 2006 (gmt 0)

Yes, all Google important engineers probably are at SES - aslong as Google blackhat friends are kept happy they dont really worry about the whitehats.

A happy blackhat does less damage therefore Google seems to do everything to please them at the conferences etc.

As to the rest of us webmasters - they dont really care.

This 38 message thread spans 2 pages: 38 ( [1] 2 > >
Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Google / Google SEO News and Discussion
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved