Indexed pages count drops yet G*bot keeps pounding

[1script]

Wonder if someone has a theory that would explain this situation:

Site gets 10,000+ pages picked up by G*bot daily, which is roughly the historical average for the site. However, the indexed pages count in both WMT Sitemaps and site:example.com operator is actually going down slowly but surely for almost a month now. Given that I have approx 50,000 pages in the sitemap (although more pages exist on the site due to pagination of the content), at this rate it appears that Google can re-cycle the entire content of the site in about 1 week. However, the number of indexed pages hovers at approx 60% of the total count and never goes higher than that. Indeed, as the title suggests, it goes down instead.

So what do you guys think can cause an erosion of indexed pages like that? Some pages get re-evaluated and deemed no longer worth keeping? Why pick them up in the first place - they might have been looked at no longer than a week ago, so an algo change would not happen that quick? Or is there a contraction of Google's own index that's going on?

Anyways, ideas on the indexing issue would be greatly appreciated!

[limoshawn]

I've been seeing the indexed pages for one of our sites fluctuating wildly for the last month, no idea why. some days google shows 70,000 - 80,000 pages indexed, some days 1,200.

[aristotle]

The total number of pages on the internet is growing rapidly. Millions of new pages are created everyday. So there are at least two possibilities:

1. Google has decided that there is an ultimate practial limit to how many pages it can keep in the index, so has establisbed more stringent requirements on what to include. This could cause some previously-indexed pages to be de-indexed because they no longer meet the new requirements.

2. As new pages are created, Google has to jettison some of the older pages from its index in order to make room for new ones.

[1script]

@aristotle: Re: growing overall number of pages in G* index

I understand as much but then every single webmaster in the world would have to report the same exact thing. Is this as common as your suggestion implies?

I do have other sites as well and I can't say that I'm particularly happy about the depth of indexing on those either. But this is the only site of mine I can think of that shows this pattern of steady decline. This site looses approx 200-400 indexed pages every day. My other sites may gain a little, then loose a little. Also the site in question is the one with the most G*bot traffic and, at least so far, the most G* referral traffic. I'm afraid continuously losing the number of indexed pages will eventually lead to limited or no G* referral traffic.

So, I guess I'm looking for any possibility that there could be local on-site factors leading to this index erosion.

Thanks!

[aristotle]

So, I guess I'm looking for any possibility that there could be local on-site factors leading to this index erosion.

If Google has introduced more stringent requirements for inclusion in its index, then some sites could be affected more than others. Obviously I don't know what the requirements are, or how they might have changed. I've seen other members here suggest several possible reasons why pages could be dropping from the index:

1. Not enough total content

2. Not enough original content

3. Not enough internal limks

4. Not enough incoming external links