Msg#: 4046086 posted 12:23 pm on Dec 19, 2009 (gmt 0)
Can anyone help. I have a site which I'm always adding 33 sitemap.xml.gz. and one sitemap.xml.
I resubmit my sitemap to google about 4 month and up until recently the 'Indexed URLs' has been going up. When I had 1,296,796 pages, google had indexed about 100,870. when i had search in google site:agentworld.com 1 mont before its show 174000 but today is show 144,000 .
The crawler access is fine, the crawl errors only 14 Short meta descriptions .
Msg#: 4046086 posted 11:50 pm on Dec 19, 2009 (gmt 0)
Welcome to WebmasterWorld!
The thing you are doing wrong is thinking that the site: search on Google is highly accurate on large website - it is not. When Google says that it is showing results 1-10 of 174,000 it is actually guessing that there are 174,000 pages. You should rely more on the numbers provided in Google Webmaster Central than the site:.
If you a million pages it is highly unlikely that Google will index all of the pages. To index a page Google needs to deem that the page is useful & unique. When you have a million pages there are bound to be some duplicate or near duplicate pages. That is why most large websites don't have 100% of their pages indexed.
Congrats on getting over 100,000 pages indexed! Hopefully they are driving traffic that is converting for you.