homepage Welcome to WebmasterWorld Guest from 54.167.179.48
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

    
sitemap.xml for a Forum - A few questions
Christa



 
Msg#: 4622487 posted 9:12 am on Nov 11, 2013 (gmt 0)

Sorry but I found it difficult to find answers to these questions:

I have a site that has a forum. My questions are:
1. Is it truly better to have sitemap.xml for a forum? If so -

2. A forum could change daily, multiple times depending on if people post/reply/register etc.
So that would mean that I would have to create a new sitemap.xml every single day? (that sounds ridiculous)
There must be a better way?

3. If I were to leave out the URL to the forum, or any URL for that matter, on my sitemap.xml (much faster to create) does this mean that search engine robots will then ignore the forum entirely and ONLY search the URLs list in the sitemap.xml?
Or they will still crawl it?

4. If I were to do a separate sitemap.xml for the forum would it still go in the root directory or the forums parent directory?
( I would list the location for it on the robots.txt)


Thank you very much for your time.

 

Robert Charlton

WebmasterWorld Administrator robert_charlton us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4622487 posted 8:37 pm on Nov 11, 2013 (gmt 0)

Hi Christa, and welcome to WebmasterWorld.

An xml sitemap really isn't necessary. Since the process of creating one involves spidering your site, though, that process in itself may serve as a helpful disagnostic for you to locate potential sources of crawling errors. In the event that your site isn't set up properly, an xml sitemap may also help Google locate pages it might not otherwise find.

xml sitemaps do not help with rankings, and they don't assure that all of your pages will be crawled or indexed. Site canonicalization, clear and correct navigation, and good inbound linking are likely to be much more helpful with regard to these factors.

If you don't know whether your site is set up well, it might be helpful to try a sitemap and see what kind of errors, if any, are reported. If you've successfully submitted a sitemap before, and you know that your site is well canonicalized, IMO it's not likely to be helpful to submit ongoing updated sitemaps. I myself wouldn't make great efforts to keep a forum sitemap current.

Here are a couple of threads I found by using Google to search webmasterworld.com which might be helpful to you, and suggest specific approaches some webmasters have taken....

Can Sitemap Errors Create a Ranking Issue?
http://www.webmasterworld.com/google/4401671.htm [webmasterworld.com]

Sitemap - change it every time I add a page?
http://www.webmasterworld.com/google/3580066.htm [webmasterworld.com]

martinibuster

WebmasterWorld Administrator martinibuster us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4622487 posted 9:25 pm on Nov 11, 2013 (gmt 0)

1. Is it truly better to have sitemap.xml for a forum?


I don't have a definitive answer because I've never installed an XML sitemap for any of my sites because I've never needed one. It is my experience that forum discussions are indexed within minutes without need of an XML site map. Which is why I've never found it necessary to create an XML site map.

2. ...So that would mean that I would have to create a new sitemap.xml every single day?


XML site map generators can create the sitemaps on their own by a scheduled rebuild. From what I understand, most site maps aren't dynamic, i.e. rebuilding everytime there's a change. That would be impractical because the crawler would have to return to reindex the site map everytime someone made a post. So it's probably more practical to set it to rebuild once a day.

They also can be configured to exclude certain forums. Make sure they're not publishing site map URLs to private forums.

3. If I were to leave out the URL to the forum... does this mean that search engine robots will then ignore the forum entirely...


URLs excluded from the XML sitemap will still be crawled because of the links pointing to them.

[edited by: martinibuster at 10:21 pm (utc) on Nov 11, 2013]

phranque

WebmasterWorld Administrator phranque us a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month



 
Msg#: 4622487 posted 9:41 pm on Nov 11, 2013 (gmt 0)

welcome to WebmasterWorld, Christa!


one advantage to a sitemap is that if you submit a sitemap to GWT there will be a report available showing the number of submitted urls and the number of indexed urls.
you'll often see complaints about the accuracy or recency of the numbers, so take it for what it's worth.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved