homepage Welcome to WebmasterWorld Guest from 54.205.241.107
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

    
How do you make search engines index ALL the urls in a sitemap file?
How do you make search engines index ALL the urls in a sitemap file?
Vrindavan

10+ Year Member



 
Msg#: 4174533 posted 7:20 am on Jul 22, 2010 (gmt 0)

How do you make search engines index ALL the urls in a sitemap file?

 

goodroi

WebmasterWorld Administrator goodroi us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4174533 posted 12:18 pm on Jul 22, 2010 (gmt 0)

the same way you make google disclose their secret ranking formula to you - you dont.

sitemaps are basically a way to suggest urls to the search engines. search engines are not obligated to index every url and rarely index every url on larger sites.

Vrindavan

10+ Year Member



 
Msg#: 4174533 posted 1:24 pm on Jul 22, 2010 (gmt 0)

What percentage indexed is normal ?

e.g. 10%, 30%, 70%, etc

or no such reference at all.

phranque

WebmasterWorld Administrator phranque us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4174533 posted 2:49 am on Jul 23, 2010 (gmt 0)

even if it were 100% you wouldn't be able to determine if it were the fact the urls were in the sitemap or other factors, unless of course some of those urls were not linked or published elsewhere.

Vrindavan

10+ Year Member



 
Msg#: 4174533 posted 4:56 am on Jul 23, 2010 (gmt 0)

>> wouldn't be able to determine

Compare before add sitemap and after add sitemap to see the difference.

phranque

WebmasterWorld Administrator phranque us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4174533 posted 7:37 am on Jul 23, 2010 (gmt 0)

correlation (or mere coincidence, for that matter) don't imply causation.
how do you know whether the discovery of a url eventually occurred through internal linking or the introduction of the sitemap unless you have isolated that factor?

Vrindavan

10+ Year Member



 
Msg#: 4174533 posted 9:05 am on Jul 23, 2010 (gmt 0)

>> don't imply causation.

agree, but it suggests "higher probability" that it has an influence if good result after the change not long.

goodroi

WebmasterWorld Administrator goodroi us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4174533 posted 12:45 pm on Jul 23, 2010 (gmt 0)

there arent good numbers to go by. google and the other search engines have said quite often that they prefer to discover urls by following links instead of submitting sitemaps. however i have seen sitemaps help large websites increase the amount of pages indexed.

i cant tell you what percentage because as phranque points out i wouldnt know if google found the url via sitemap or via links or possibly even via toolbar usage.

sitemaps can help certain websites with indexing issues. imho most websites are better served by making sure all pages are internally interlinked and developing inbound links from many external websites. this will help indexing and more importantly improve the rankings of the pages that are indexed.

dont forget that unique content can also have a significant impact on your indexing %. if you have 1 million pages and they are all identical, then search engines will not want to index all of your pages. good luck

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved