How do you make search engines index ALL the urls in a sitemap file?
How do you make search engines index ALL the urls in a sitemap file?
Vrindavan
7:20 am on Jul 22, 2010 (gmt 0)
How do you make search engines index ALL the urls in a sitemap file?
goodroi
12:18 pm on Jul 22, 2010 (gmt 0)
the same way you make google disclose their secret ranking formula to you - you dont.
sitemaps are basically a way to suggest urls to the search engines. search engines are not obligated to index every url and rarely index every url on larger sites.
Vrindavan
1:24 pm on Jul 22, 2010 (gmt 0)
What percentage indexed is normal ?
e.g. 10%, 30%, 70%, etc
or no such reference at all.
phranque
2:49 am on Jul 23, 2010 (gmt 0)
even if it were 100% you wouldn't be able to determine if it were the fact the urls were in the sitemap or other factors, unless of course some of those urls were not linked or published elsewhere.
Vrindavan
4:56 am on Jul 23, 2010 (gmt 0)
>> wouldn't be able to determine
Compare before add sitemap and after add sitemap to see the difference.
phranque
7:37 am on Jul 23, 2010 (gmt 0)
correlation (or mere coincidence, for that matter) don't imply causation. how do you know whether the discovery of a url eventually occurred through internal linking or the introduction of the sitemap unless you have isolated that factor?
Vrindavan
9:05 am on Jul 23, 2010 (gmt 0)
>> don't imply causation.
agree, but it suggests "higher probability" that it has an influence if good result after the change not long.
goodroi
12:45 pm on Jul 23, 2010 (gmt 0)
there arent good numbers to go by. google and the other search engines have said quite often that they prefer to discover urls by following links instead of submitting sitemaps. however i have seen sitemaps help large websites increase the amount of pages indexed.
i cant tell you what percentage because as phranque points out i wouldnt know if google found the url via sitemap or via links or possibly even via toolbar usage.
sitemaps can help certain websites with indexing issues. imho most websites are better served by making sure all pages are internally interlinked and developing inbound links from many external websites. this will help indexing and more importantly improve the rankings of the pages that are indexed.
dont forget that unique content can also have a significant impact on your indexing %. if you have 1 million pages and they are all identical, then search engines will not want to index all of your pages. good luck
