homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

Google Sitemap Index
Sitemap Index problem

5+ Year Member

Msg#: 4046086 posted 12:23 pm on Dec 19, 2009 (gmt 0)

Can anyone help. I have a site which I'm always adding 33 sitemap.xml.gz. and one sitemap.xml.

I resubmit my sitemap to google about 4 month and up until recently the 'Indexed URLs' has been going up.
When I had 1,296,796 pages, google had indexed about 100,870.
when i had search in google site:agentworld.com 1 mont before its show 174000 but today is show 144,000 .

The crawler access is fine, the crawl errors only 14 Short meta descriptions .

What am I doing wrong?




WebmasterWorld Administrator goodroi us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

Msg#: 4046086 posted 11:50 pm on Dec 19, 2009 (gmt 0)

Welcome to WebmasterWorld!

The thing you are doing wrong is thinking that the site: search on Google is highly accurate on large website - it is not. When Google says that it is showing results 1-10 of 174,000 it is actually guessing that there are 174,000 pages. You should rely more on the numbers provided in Google Webmaster Central than the site:.

If you a million pages it is highly unlikely that Google will index all of the pages. To index a page Google needs to deem that the page is useful & unique. When you have a million pages there are bound to be some duplicate or near duplicate pages. That is why most large websites don't have 100% of their pages indexed.

Congrats on getting over 100,000 pages indexed! Hopefully they are driving traffic that is converting for you.

Global Options:
 top home search open messages active posts  

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved