homepage Welcome to WebmasterWorld Guest from
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Become a Pro Member

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

Google Webmaster Tools w/Sitemaps
Not all the pages in sitemap are indexed

 2:23 pm on Dec 7, 2009 (gmt 0)

I have search WebmasterWorld for an answer to this question, but haven't found one.

I have submitted a sitemap in Google Webmaster Tools which consist of 13 urls, but only 12 are indexed. So I kind of have a two part question here.

GWT used to have a link, "What Google sees" or something like that, but it is gone now. How am I able to tell what urls it is indexing and which ones it isn't?

Also what would be some of the usual causes for GWT to not index 1 or 2 urls?

I have quite a few sites and half of them, all the submitted urls are indexed and half of them are not indexing 1 to a few urls.




 1:49 pm on Dec 8, 2009 (gmt 0)

This is an impossible search! Every time I search for "show urls not indexed" and many other variations... All I get are results of people complaining about none of their pages being indexed. I don't care about that! I want a report of what urls are being indexed! Can no one answer this?


 3:37 pm on Dec 8, 2009 (gmt 0)

You don't even need GWT for this. Simply go to Google and enter "site:example.com" without the quotation marks and subsiting example.com with your domain name. That will display the 12 pages that Google has indexed and you should be able to figure out which one of your 13 urls are is missing from the results.

Please remember that sitemaps are simply another way to help search engines better discover your website. Sitemaps DO NOT guarantee the search engines will index your content. To increase the probability that the search engines will index your urls you should make sure each url has significant unique and useful content plus a significant amount of internal and external link popularity.

If your pages are deemed to be too similar to each other or too similar to pages already published on another website the search engines will likely not index them. If your pages have little or no link popularity the search engines will take that as a sign the pages are not very important. If the pages were valuable then more people would be linking to them.

It may be more beneficial to you if focus your efforts on making sure your content is unique and useful. The higher quality the content is the higher the chances it has to attract links and people will be more likely to link to you when you ask for links.


 4:16 pm on Dec 8, 2009 (gmt 0)

Wow thanks for the answer. And as usuall the most obvious answer.

I definitely have the "unique" factor down, and there are plenty of quality external links to these sites (vacation related, so plenty of travel sites link to them). They also usually rank if not 1st, very high in SERPs.

I had this question because I was putting together a robots.txt and It made me question the fact of if I even wanted these un-indexed pages to be indexed at all? Do I want to remove them? Then I thought Well I don't even know what the pages are. But now I do. Thanks!


 4:25 pm on Dec 8, 2009 (gmt 0)

So Here is what I found... Which raises another question.

The page not being indexed is the example.com/index.htm. So within my sitemap I have the index.htm listed as well as the example.com/.

Should I even have this listed in my sitemap at all, considering they are both the same? I guess what I am trying to ask is if it pointless to list both?


 11:02 am on Dec 9, 2009 (gmt 0)

It is generally not helpful to list duplicate content in a sitemap.

Global Options:
 top home search open messages active posts  

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved