Welcome to WebmasterWorld Guest from 23.23.46.20

Number of pages indexed drops - a lot and very fast.

Why...

   
4:36 am on Feb 27, 2006 (gmt 0)

10+ Year Member



My page index has decrease more than 100% since BigDaddy hit. And this morning I found that it continue to drop from 28,600 to now 7,200 pages.

Anyone know the reasons why? I just change the redirection method within my site from 302 to 301. Also already create Google sitemap and status of sitemap is OK.

I'm pretty sure that I didn't use any bad technique to fool GG, everything that I do was all white technique.

Anyone has the same problem? I'm now is very concern about this issue.

Thanks for your suggesstion in advance.

4:55 am on Feb 27, 2006 (gmt 0)

WebmasterWorld Senior Member tedster is a WebmasterWorld Top Contributor of All Time 10+ Year Member



I'm hearing a lot of reports like this right now, as Google's Big Daddy infrastructure is in the middle of rolling out -- but not yet stable. Here's the important question -- is your traffic affected? If it isn't, I'd just wait it out.

But if your traffic is affected, and pages that used to get search traffic now are not, or are severely depressed, then it's time for a through check-up of your site. These threads have lots of detail about areas to inspect for problems:

Checklist for Sudden Drops in Rank [webmasterworld.com]
Dropped from Google - a checklist to find out why [webmasterworld.com]
Dropped Site Checklist [webmasterworld.com]

5:57 am on Feb 27, 2006 (gmt 0)

10+ Year Member



I'm experiencing the same things too. My pages began to drop from BD database since the beginning of last week, but no problem with other non-BD databases. Because BD is spreading to more and more datacenters, my traffic began to decline.
6:47 am on Feb 27, 2006 (gmt 0)

WebmasterWorld Senior Member tedster is a WebmasterWorld Top Contributor of All Time 10+ Year Member



Questions for Cyclob, sit2510, and anyone else who is seeing this problem --

1. Are you seeing Mozilla Googlebot crawling your pages?
2. Are you fully in the Sitemap program?

I know that you, Cyclob, said that your sitemap was verified -- but are things progressing from their? Are you seeing any statistics?

7:41 am on Feb 27, 2006 (gmt 0)

5+ Year Member



Same is happening to me. site: returns 11,900 on regular and 255! on bigdaddy. I use sitemaps in full and get statistics. The site is 100% clean. Traffic is down by 50% as many pages lost about 100 spots on serps.

I am clueless...

7:48 am on Feb 27, 2006 (gmt 0)

10+ Year Member



Also for me, huge drop in pages indexed, a lot of my most heavily weighted and link to pages thrown out in SERPS and in site:. Traffic down a bit because of it. Actively use sitemaps, got stats, it appears to work fine.
10:10 am on Feb 27, 2006 (gmt 0)

10+ Year Member



Huge drop in pages indexed in BD for me.

Yes - Mozilla Googlebot crawls the site
No - Not in Google sitemaps program

11:35 am on Feb 27, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



No drastic change in traffic, it's up slightly if anything, but I'm also seeing a reduction in page counts. But before now Google was over-reporting my count, so a website with around 10 000 pages was reported with a site: index of about 25 000.
11:48 am on Feb 27, 2006 (gmt 0)

10+ Year Member



Ouch - all the pages that were removed from my site have gone to Pagerank N/A.
11:51 am on Feb 27, 2006 (gmt 0)

10+ Year Member



I have a weak indication on the Mot-Bigdaddy-datacenters for months. Only the Mozilla-Bot is very active. On Bigdaddy my big shopping portal has been indexed fully and I have 4 times as much Traffic there. But I have lost URLs strongly on Bigdaddy since Thursday.
12:15 pm on Feb 27, 2006 (gmt 0)

WebmasterWorld Senior Member billys is a WebmasterWorld Top Contributor of All Time 10+ Year Member



1. Are you seeing Mozilla Googlebot crawling your pages?
2. Are you fully in the Sitemap program?

I know that you, Cyclob, said that your sitemap was verified -- but are things progressing from their? Are you seeing any statistics?

1. - Very active Mozilla Googlebot and GOoglebot

2. - I've been using Sitemap for about 2 months now. (verified and seeing statistics)

I think there is going to be a 2 phase rollout of BD. Infrastructure first, followed by an index update. I rank well on BD and non-BD DCs with the execption I've got roughly 220 pages in BD and 1,030 in non-BD centers.

12:37 pm on Feb 27, 2006 (gmt 0)

5+ Year Member



BillyS,

How many visits daily do you get from Mozilla Googlebot? I am curious what is "normal" for one day.

12:43 pm on Feb 27, 2006 (gmt 0)

10+ Year Member



>>>>I think there is going to be a 2 phase rollout of BD. Infrastructure first, followed by an index update.

I sure hope you are right BillyS

2:47 pm on Feb 27, 2006 (gmt 0)

10+ Year Member



I am seeing the same thing. Over the last week I have seen pages indexed go from 500,000 to 100,000. My site is prett much an authority site. Some decrease in traffic has been seen due to not having all the search terms in google any longer. I do rank higher for most that are there, but so many pages that were indexed simply are gone.

Not sure what to think.

My other VB forum which is pretty much set up the same has had an increase in pages indexed which is strange

3:03 pm on Feb 27, 2006 (gmt 0)

10+ Year Member



All my sites have improved over this last week (one even jumped from 400 pages to 10000 on BD)..

I am still down on what the non-BD DC's are showing, but its heading in the right direction.

I am sure Google will get to all of us guys ;-)

3:48 pm on Feb 27, 2006 (gmt 0)

5+ Year Member



Could you please specific the IP of that Datacenter.

Today I checked on 64.233.183.99 and 64.233.183.104 the result came up from 11,000 to 142,000 unfortunately, now the result roll back to normal (11,000)

Any suggestions?

4:44 pm on Feb 27, 2006 (gmt 0)

WebmasterWorld Senior Member billys is a WebmasterWorld Top Contributor of All Time 10+ Year Member



>>How many visits daily do you get from Mozilla Googlebot? I am curious what is "normal" for one day.

For my site, Googlbot is more a hit and run bot. It appears, pulls a lot of pages then goes away for a bit. So I'm not sure what an average day looks like.

Earlier today I had Mozilla Googlebot grab 350 page (35% of the site) over a 3 hour period. After seeing this thread I just checked my logs and Googlebot 2.1 has grabbed about 300 pages over the last hour or so.
Up until today:

Googlebot 2.1 = 3,043 pages
Mozilla Googlebot = 2,559 pages

Despite all this spidering, I continue to lose pages in BD's index. From a high of 1,020 pages I'm now down to 312 (with about 100 of those being supplemental) - an all time low for the last two years.

Came from my post here: [webmasterworld.com...]

5:11 pm on Feb 27, 2006 (gmt 0)

10+ Year Member



If I am not using Google Sitemap program, am i facing any sort of penalty for which i noticed that all my internal index pages disappeared from Google.
As far GoogleBot is concerend, it is visiting everyday.
I dunno wat to do and how to react to the present situation.
7:05 pm on Feb 27, 2006 (gmt 0)

10+ Year Member



Traffic is down about 25% since last Thursday.

I'm seeing a drop of indexed pages on BigDaddy DCs of anywhere from 65% to 80%.

The Mozilla Gbot crawls about 30k pages a day and have been for the last month or so.

And I don't use sitemap.

7:50 pm on Feb 27, 2006 (gmt 0)

10+ Year Member



>>>>I think there is going to be a 2 phase rollout of BD. Infrastructure first, followed by an index update.

So... how can we tell when BD starts to do an index update? Thus telling if you have been dropped for good or not?

BD has dropped pages on us it seems every day. No change for us (in SEO actions) than before BD. I have been told by many to sit and wait, but I want to know what to look for.

2:58 am on Feb 28, 2006 (gmt 0)

10+ Year Member



Also noticing many pages that have had no. 1 rankings for over a year, Pr5 etc - now lost their cache.

Yes when I check the other datacenters, many of them have current caches. Glad to hear that others have the same issue. Hope its just some bug that Google will be fixing as it makes no sense otherwise.

8:56 am on Feb 28, 2006 (gmt 0)

10+ Year Member



Questions from tedster

1. Are you seeing Mozilla Googlebot crawling your pages?

Yes, Mozilla Googlebot does crawl the missing pages as well as the normal bot. In non-BD, the missing pages are still there with latest cache of 23 or 24 Feb 06, but those of the very same pages are not in BD datacenters.

2. Are you fully in the Sitemap program?

No, it is not in the Sitemap program.

I'm not sure whether it is a huge bug in Big Daddy that accidentally lose pages or what else...If so, then everyone is at risk of having Google fumbles its ball with our pages disappear.

4:15 pm on Feb 28, 2006 (gmt 0)

WebmasterWorld Senior Member tedster is a WebmasterWorld Top Contributor of All Time 10+ Year Member



BD is not stable, and has a lot of funny data right now. If your pages are getting spidered, then I'd say hold on for a bit before hitting the panic button. You are far from the only one seeing this.
12:58 am on Mar 1, 2006 (gmt 0)

10+ Year Member



Today I checked on 64.233.183.99 and 64.233.183.104 the result came up from 11,000 to 142,000 unfortunately, now the result roll back to normal (11,000)

I am in a similar situation for my one big resource website. I am going to wait it out.

6:06 am on Mar 1, 2006 (gmt 0)

5+ Year Member



Are there anyone to point out on [64.233.183.104...] Is it really BD or it's rubbish?
7:46 am on Mar 1, 2006 (gmt 0)

10+ Year Member



In an email from google:

New sites, changes to existing sites, and dead links will all be
noted after our next crawl, which will be completed soon.

Could be the second part of big daddy.

9:06 am on Mar 1, 2006 (gmt 0)

10+ Year Member



Intresting Nick0r

One report from SES has Matt Cutts saying 6 weeks for Big Daddy roll out - which is what he said 4 weeks ago.

So how soon is soon for the next crawl completion I wonder.

3:58 pm on Mar 1, 2006 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



hmmm. Yes Our main site has 274,000 pages and now it is showing 74,000. Also a lot of well linked pages have gone from the serps. I have seen this before, we came back although it is uncomfotable waiting for it to get back ot normal. Hope this big daddy stuff is implimented sooner rather than later.
7:35 pm on Mar 1, 2006 (gmt 0)

10+ Year Member



Crush, I am in the same exact situation as you. Numbers are almost exact. Was 290k indexed, now only 72k.

Crossing my fingers. We are an authority site in our niche.

4:58 pm on Mar 2, 2006 (gmt 0)

10+ Year Member



Nick0r, above you mention the "second part of BigDaddy" I was under the impression that BigDaddy was a gradual roll-out to the various data centers, not a distinct multi-part update like Jagger. Did you read otherwise somewhere?
This 32 message thread spans 2 pages: 32
 

Featured Threads

My Threads

Hot Threads This Week

Hot Threads This Month