Site: operator says most of my indexed pages are "omitted"

[Marty3454]

Hi,

I have been having a problem with google for some time.
Google doesnt seem to be indexing all the pages on my site - it only indexes about 600-700 pages when there is "at least" 9000 pages on the site - each with unique content.

If i do a "site" search in google, it says there is around 80,000 pages but when i try and move to the end of the results using the pager i get the message:

"we have omitted some entries very similar" message

The pages are not simular at all (the page template is minimal), and i have plenty of inbound links from different Ip addresses (and related sites). ~ 847 inbound links using the "link" operator.
My site also follows all the google guidelines.
My site also has plenty of internal links to each of the pages.

any help would be greatly appeciated, im about to give up on this issue because it has been around for ~ 6 months and its driving me insane.
The problem doesnt exist with yahoo, so im really confused.

let me know if you need the domain name to check the site.

thanks,
martin

[Quadrille]

First, never trust Google 'webmaster' seraches; they do not - ever - give full or accurate info. For better info, use Google webmaster tools.

While your pages may have unique content, do they have unique <title>s and meta descriptions? that's a common lack that can prevent full listing of your pages.

What matters, however, is how 'real' searches work - test some 'unlisted pages' with keyword searches, and searches for 'unique strings of text' from the page - you may find things aren't so bad!

But - standard advice for any Google problem - do check ALL your outgoing links, and junk any that are paidlinks, bad neigborhoods, unrelated reciprocals, etc., etc.

[Marty3454]

Hi,

Thanks for the speedy reply.

The titles and descriptions are unique, there is only a hand full of duplicate ones (~100 or so) so im sure thats not the problem.

Im fairly certain they are being dropped off the index because my traffic has gone down.
There was a moment (about a month ago) where this problem went away and traffic went up, but now the problem has come back and traffic has gone down again. I tend to trust the site search more than google webmaster tools because apparently according to webmaster tools everything in my sitemap is indexed (my site map has main structure and the ~9000 or so unique pages i was mentioning)

I dont have any bad outgoing links, they are all job related sites mantainted by my company (but most sites sit on different ips)

thanks for the feedback.

[TerrCan123]

I don't think this may be a "problem" in the sense many sites only have some pages indexed, even the sites Google loves. Another site I checked had the same number of pages indexed as mine 600 or so and [about 35%] and they were a top "favorite" site.

I just think this is the new Google but it's only my opinion. I just doubt we can get them all indexed or ever will again, the days of 2005 are past and may never return.

If you wonder though check some related sites and see how they are indexed to compare.

[Marty3454]

Hi,

Other "job search" (which is what my site is) sites index much more pages and have less inbound links. No only that, they are not very search engine friendly (ie dont have clean urls, have duplicate content all over the place, and are poorly designed in general)

Thats another reason why im convinced im being penalised for something i have no idea about.

- martin

[tedster]

Check in your Webmaster Tools account for any issues with your meta descriptions. On lower PR pages, insufficient uniqueness across meta descriptions (or the complete absence of them) can be one factor in shipping those urls off to the back regions.

[edited by: tedster at 5:21 am (utc) on Sep. 1, 2008]

[Marcia]

>>"we have omitted some entries very similar" message

If you haven't already, copy and paste unique snippets from your "omitted" pages into a Google search box, in quotes, and do a serious hunt for scraped copies of your pages.

[Marty3454]

Hi,

I have done that already. Not all pages show up :(
The majority of the job displays should be showing up because they all have unique content.

martin

[experienced]

similarly there are sites that had approx 20k to 100k pages indexed but when we do a site search sometiem they return only 20 to 30 pages.. and there is no link for ommited results.. what could be the cause for this...?

even i have a site with over 2000 pages indexed and since couple of days there are only 17 pages can be seen with no ommited link.. similarly one site has 85K pages where approx 39k was indexed and today it has only 13 pages...

Anybody has the similar issue and has got the solution..?

would like to share
thanks a lot