Big difference between GWT and site: operator numbers

[virtualreality]

I have a website with couple of thousands pages - i dont sell anything. all is content. i also have google sitemaps (xml, rss, html and txt). i have a rss feed as well. in my webmasters tool acount my sitemap says Google had indexed 1160 pages out of 1700 but when i do "site:www.myweb.com" in Google i see only 260 results. Why is that?

[g1smd]

The public SERPs lag what they have actually spidered by at least several days.

I helped a new site go online a few months ago... and the number of pages showing in the SERPs rose slowly during that time... 1... 7... 9... 12... 15... 16... 24... 30... 36... and so on.

Additionally, different datacentres had a different number of pages indexed, with a 30% difference between the best and the worst.

The number of pages listed in the on-site Google Custom Search for the site for a site:domain.com search were always ahead of the normal SERPs by some 5% to 20%.

GWT flagged some of the meta descriptions as "too short" (in the "Content Analysis" section) .

In the normal SERPs (for a site:domain.com search) those same pages were listed with the first 20 words from the page showing as the snippet instead of the meta description text.

There were several more with the same snippet text that were not yet flagged in WMT as having a meta description that was too short, and we fixed those meta descriptions before they were flagged as sub-standard.

There is one interesting thing about site:domain.com counts, that has started in the last few days.

I have recently found site:domain.com over-reporting the number of indexed pages by about 15 when viewing the first page [1 to 10 of about 85] of the SERPs. The site only has 70 pages.

However, a site:domain.com/* search (which shows Supplemental Results) returns 15 results.

In this case, all 15 URLs that appear as Supplemental Results also appear in the list of 70 normal results.

The site has 70 pages. All pages are newly online in the last few months.

There are no URLs that used to exist that no longer do so. All pages are new, but several pages have been edited or added to several times during the last few months.

So, the only type of Supplemental that there can be here, are Historical Supplemental** Results.

So, it seems to me that the initial estimate is for all of the results in their index. The number shown is therefore normal results plus supplemental results and that total is before the list has had duplicates removed.

**see my thread from 2006 for what "Historical Supplemental" means.

[tedster]

GWT flagged some of the meta descriptions as "too short" and in the normal SERPs (for a site:domain.com search) those same pages were listed with the first 20 words from the page showing as the snippet instead of the meta description text.

Nice observation. Sounds like this GWT report might come, at least in part, from the snippet team's algo!

[g1smd]

Yes, Tedster, Google gives a number of clues in a site:domain.com search as to things that are sub-optimal. The best clue always used to be the Supplemental tag. Although that has gone, there are still many others if you look hard enough.

.

Back to the question original , in more detail.

Does your results set end with "In order to show you the most relevant results, we have omitted some entries very similar to the nnn already displayed. If you like, you can repeat the search with the omitted results included."?

What happens when you click that?

What happens when you select 100 results per page, instead of the default of 10 results?

Do you have multiple pages showing the same title and/or same snippet in a site:domain.com search?

.

Do you get a different results set for each of these searches?

site:domain.com
site:www.domain.com
site:domain.com -inurl:www

[g1smd]

How are you getting on with this?

[tomda]

Nice post g1smd...

Doing search for site:www.domain.com and...

It shows 445 results at first page...

Then, when I go on page 16, it shows 150 results and the "In order to show you the most relevant results, blah blah"

I click on it and get 182 results ! The last page being page 19.

I have 32 duplicate URLs, but they have been removed using the GWT last week... I do not know why they got indexed because I use robots.txt. Anyway, went to GWT, it is written "Removed".

I guess I'll wait another month ?
And why showing 445 results instead of 150 ?

[g1smd]

You'll have to work that out yourself. There are some clues in what they list though.

Try both site:www.domain.com and site:domain.com -inurl:www to be sure.

[tomda]

I think I am OK... I'll wait a bit.

site:www.domain.com - 459 results
site:domain.com -inurl:www - 0 result

Thanks

[g1smd]

OK. That's a GOOD sign, and something that less than 1% of webmasters have mastered.

[g1smd]

*** However, a site:domain.com/* search (which shows Supplemental Results) returns 15 results. ***

Correction: the /* search shows non-Supplemental results.