Inflated Page Counts in Google

Hitting the 1,000 page mark caused quite a jump...

[BillyS]

My website has been nearing a milestone for the past couple of weeks as the site approached the 1,000 page mark in Google. I had been watching this closely (with some pride) as the number in Google's index climbed.

Over the past couple of days, the number stood at 999 using the site:www.foo.com command. Today I noticed something interesting. The same command that returned 999 pages over the last several days now shows 9,140 pages. Quite a jump. Now I understand why everyone's been talking about inflated page numbers. I was wondering if this happened only for sites in excess of 1,000 pages in Google's index. For someone that just crossed this milestone, that was my experience.

[DerekH]

No, I don't think it's anything to do with crossing a thousand. I more suspect that it's just a guesstimate based on some primitive calculation.
One of my sites has the same sort of over-estimate, but as you go through the Google SERPS, the estimate gets more and better refined each time you ask Google to display the next page of "pages from my site".

Hardly surprising, since one of Google's aims is to present relevant information *quickly*.

DerekH

[steveb]

At the 1000 page milestone something definitely happens, or at least makes stuff more likely to happen. I don't know what, but I do know that for sites less than 1000 pages Google is normally almost exactly accurate with page counts (including URL only and Supplementals) while over 1000 pages Google seems never really close to accurate.

Are there people out there with 1126 page websites who display 1126 pages via the site: command?

Having those inflated page numbers obviously has no positive, so staying below 1000 pages appears to be something to definitely aim at, if only to track how many Supplementals and other screwed up listings Google has for you.

[BillyS]

There are lots of moving parts here, so I don't want to be paranoid about things. I'm not sure if or how they are connected, but as I passed the 1,000 page mark, the site dropped off the Google map. It's almost like it went back into the "sandbox."

The telltale sign for me is my name (uncommon last name). After coming out of the sandbox in June / July, I used to rank #1 for my own name (actually held the #1 and #2 spots. Now I've got about ten sites above me.

The site grows at roughly 5 pages a week for the last 7 months, but I'm wondering if the 1,000 page threshold is another flag for Google. Certainly if Google "thinks" the site is 9,100 pages, then it might be suspicious of the site growing by 8,000 pages overnight.

[herb]

Have a site with about 12,000 pages. Google for 4-5 months reported 24,000 and was sending visitors. Beginning of this month it thinks we have 72,000 and stopped sending visitors.

[my3cents]

my site has about 550 pages, google has been showing 1090 - mostly because it has the www and non-www versions of the website, I can actually see 949 pages of the site indexed, almost all of it supplimental.

I have had a rewrite for the non www pages for over 6 months, but google recently brought back in cached versions of non-www pages from last year.

Seems like everytime they start indexing the site correctly, they bring in the non-www pages or other very old versions of the pages with various url. G index is showing some urls for pages I have never had.

[BillyS]

By the way, at least HotBot can count. Checking in Hotbot indicates that Google has exactly 1,000 pages for my website - not 9,100.

[AndyA]

I wonder if there's anything to the fact that when doing a site: search, Google has both www and non-www pages listed. The count is the same for my site, and I did a 301 earlier this year to permanently redirect everything to non-www, since Google had more of those pages lised (probably due to internal linking of the site).

Prior to doing the 301, Google showed different counts for www and non-www. The current count is pretty close on my site.

[linkjack]

I can confirm this.

My site was showing 8,232 recently (a reasonably accurate count)

But now it shows 56,998 which is unreasonably high, I never produced more than 9000 pages for sure.

My link count has dropped to record lows though, basically my links disappeared.

[followgreg]

I've seen that also and it's totally unrealistic for many sites.

Since this morning I see a wave of "spupid" rankings changes going with an inflated page count...

[texasville]

awww...now this is getting ridiculous! I have a small site. Now google has gone and drug up pages I 404'd a long time ago. Inflated my page count from 38 pages to 44 pages with cache dates from 2004. Smells like either a "who's 'index' (insert a more graphical term) is bigger" or we are on the edge of a major update. All the signs show update. Google gone quiet. Old cache dates. Or maybe Sergey Brin is so teee'd off about yahoo's claim that he's dredging up everything he can to show his is bigger.
So, is this an update or are we all serp viagra? Whichever, I wish they would stop picking on us little guys.

[reseller]

Hi Folks

This inflations in page numbers, is it happening ONLY for the penalized sites/ sites lost ranking on the serps, or its happening also to sites that rank well on the serps?

[phantombookman]

This inflations in page numbers, is it happening ONLY for the penalized sites/ sites lost ranking on the serps, or its happening also to sites that rank well on the serps?

I have a site that is inflated from 1,200 actual to over 10,000. Not penalised and never (touch wood) affected by updates.

As with others, all my sub 1000 page sites have accurate page counts.
This started quite a while ago and the page count has risen 2 or 3 times since

[reseller]

phantombookman

Thanks for feedback.

>>I have a site that is inflated from 1,200 actual to over 10,000. Not penalised and never (touch wood) affected by updates.<<

So that "inflation" is part of business as usual for Google now!

>>This started quite a while ago and the page count has risen 2 or 3 times since<<

And that enables Google to tell Yahoo; Mine is 3 times bigger than yours ;-)

[dataguy]

As of this morning, my inflated number have been cut approximately in half by Google. Some sites a little more, some a little less.

I wish they would forget about the page count and just send me more visitors!

[phantombookman]

Hi Reseller
it started at least 6 months ago I believe. I check my page count just to see if new content is indexed. My heart stopped when I saw it at approx double!
My first thought was www and non www canonical problems (no 301, windows server without ht access)
It jumped twice since to about 5k and last week or so since the latest update/non update literally doubled up to over 10k

It seems strange, the reasons appear obvious but the ruse is so pathetically crude it is bound to get out into the broader public domain and will, presumably, be quite embarrassing for them

[wormsitesAdmin]

One of my main sites was showing 19,455 pages indexed with google, and suddenly dropped to 110. The only thing I can think of that would have caused this is that I have a total of 92 domains registered, and like a dummy, I put redirects on about 80 of them pointing to this site.. I have removed the redirects and will let you all know what happens to my page listings with google..

[BillyS]

Has anyone seen a return to the "sandbox" after getting inflated page counts or a temporary decrease in traffic?

It's a bit confusing to me, but here is my logic...

Google knows the actual page count of any site in its index - it's just a calculated value (actually just a count). But why doesn't it show the real value? Is it too lazy to calculate the value and uses an estimate instead?

But how else is this estimate used? Clearly if it stored the real value, then it would show that in the site:www.foo.com query. So perhaps when a site gets large enough it uses an algorithm instead of actual count - perhaps to save CPU cycles. But if this estimate is now used as the "actual" for a site, then perhaps a penalty can kick in for getting too big too fast. Just a thought…

If I'm wrong, then of course Google should explain why it used an "inflated" page count when it could just use an actual value it already has stored somewhere. Personally, I'd rather believe the first explanation (I'm not big on conspiracy theories).

[BillyS]

Anyone suffering from inflated pages and a drop in rank seeing an improvement here:

216.239.59.99

[phantombookman]

Anyone suffering from inflated pages and a drop in rank seeing an improvement here:
216.239.59.99

No still the same for me, all sites with over 1,000 pages show approx 10x inflation.
Under 1k pages, very accurate

[netmeg]

This inflations in page numbers, is it happening ONLY for the penalized sites/ sites lost ranking on the serps, or its happening also to sites that rank well on the serps?

I have a personal hobby site, that, as of the Bourbon update, ranks extremely well in the serps - pretty much #1 for all the words/phrases I want it to rank for. The site has ONE page; it's database driven, and the search results actually show on the index page. site:www.mysite.com returns 618 pages - so it looks like Google is seeing every database record as a separate page - except that there are only 390 records in my database, so I'm not sure where it's getting the extra pages. It's not a question of www vs. non-www - the non-www results show a different (lesser) number of pages altogether.

Weird.

[my3cents]

I do see an improvement on 216.239.59.99

It looks like that DC does not have my non-www pages index, but shows tracking urls and a couple of other old dead pages, still a lot more accurate than I have seen lately, though almost every page is supplimental.

Maybe they are moving in the right direction...

[twebdonny]

Ours went over 18K, about 4X the accurate amount.
I have a feeling this one one of the factors that is
creating a filter or penalty against our main page,
index.htm. Of course, no feedback from G on this
one either, yet our traffic is almost nonexistent
from G now, as our rankings on all pages in the SERPS
continue to drop without any reasonable explanation.

[arubicus]

"No still the same for me, all sites with over 1,000 pages show approx 10x inflation.

Under 1k pages, very accurate"

I see the same thing with our sites. On this main site (over 1000 pages - 5 years old) the directory structure has changed at a few points in time, url's have changed (were 301's and now 404'd or 410'd, both www/non-www pages were indexed at one time (although the fix has been in place for a couple of years), Google creating extra pages from 301 external redirects (since been removed but pages remain), screwed up links to our site... We show an inflation of about 8x

The newer sites have not had any url changes, screwed up links, redirects, removed page, www/non-www fix been in place from the beginning... These sites are under 1000 pages. Zero inflation for these sites.

[wiseapple]

We are at about 10X.

[steveb]

216.239.59.99, no improvement on first look.

I 301ed enough pages to get below 1000, and magically Google now correctly lists the pages on that site almost exactly right (a few non-existent Supplementals included) rather than the 10x number when I was over 1000.

No ranking improvement though, which may be the scary part.

[2by4]

This is a google indexing bug. Steveb, you're not right, sorry, I have a site listed at roughly 50 times its current actual page count, significantly below 1000 real pages, and before this bug triggered, google had it at well below one thousand pages indexed.

I know at least one cause that will trigger this bug, without any doubt at all, but it cracks me up to watch google mess up this much so I'm going to keep it to myself for now, but I've verified it on a wide range of sites, and it's always triggered if this one condition is present. Other conditions may also trigger the bug, but his one definitely does.

[poppymccool]

I have a site that dropped off the Google shelf earlier this week just after I submitted a site map for it. I submitted a site map for two other sites that are doing fine.

Now I looked at the inflation and have more than 38,000 pages in the index. That is just not possible since it is a blog with less than 2000 entries. Even including comments and other stuff it wouldn't be 20 pages for each entry.

And I don't get a better result on that other server you mentioned.

[poppymccool]

2by4 - It'd be nice if you shared it, since this inflation may be a reason why sites have dropped off the front-line SERPs.

[2by4]

Oh, sorry steveb, looks like you found another trigger for this bug. Can anyone say 'beta'?