Msg#: 4622487 posted 9:12 am on Nov 11, 2013 (gmt 0)
Sorry but I found it difficult to find answers to these questions:
I have a site that has a forum. My questions are: 1. Is it truly better to have sitemap.xml for a forum? If so -
2. A forum could change daily, multiple times depending on if people post/reply/register etc. So that would mean that I would have to create a new sitemap.xml every single day? (that sounds ridiculous) There must be a better way?
3. If I were to leave out the URL to the forum, or any URL for that matter, on my sitemap.xml (much faster to create) does this mean that search engine robots will then ignore the forum entirely and ONLY search the URLs list in the sitemap.xml? Or they will still crawl it?
4. If I were to do a separate sitemap.xml for the forum would it still go in the root directory or the forums parent directory? ( I would list the location for it on the robots.txt)
Msg#: 4622487 posted 8:37 pm on Nov 11, 2013 (gmt 0)
Hi Christa, and welcome to WebmasterWorld.
An xml sitemap really isn't necessary. Since the process of creating one involves spidering your site, though, that process in itself may serve as a helpful disagnostic for you to locate potential sources of crawling errors. In the event that your site isn't set up properly, an xml sitemap may also help Google locate pages it might not otherwise find.
xml sitemaps do not help with rankings, and they don't assure that all of your pages will be crawled or indexed. Site canonicalization, clear and correct navigation, and good inbound linking are likely to be much more helpful with regard to these factors.
If you don't know whether your site is set up well, it might be helpful to try a sitemap and see what kind of errors, if any, are reported. If you've successfully submitted a sitemap before, and you know that your site is well canonicalized, IMO it's not likely to be helpful to submit ongoing updated sitemaps. I myself wouldn't make great efforts to keep a forum sitemap current.
Here are a couple of threads I found by using Google to search webmasterworld.com which might be helpful to you, and suggest specific approaches some webmasters have taken....
Msg#: 4622487 posted 9:25 pm on Nov 11, 2013 (gmt 0)
1. Is it truly better to have sitemap.xml for a forum?
I don't have a definitive answer because I've never installed an XML sitemap for any of my sites because I've never needed one. It is my experience that forum discussions are indexed within minutes without need of an XML site map. Which is why I've never found it necessary to create an XML site map.
2. ...So that would mean that I would have to create a new sitemap.xml every single day?
XML site map generators can create the sitemaps on their own by a scheduled rebuild. From what I understand, most site maps aren't dynamic, i.e. rebuilding everytime there's a change. That would be impractical because the crawler would have to return to reindex the site map everytime someone made a post. So it's probably more practical to set it to rebuild once a day.
They also can be configured to exclude certain forums. Make sure they're not publishing site map URLs to private forums.
3. If I were to leave out the URL to the forum... does this mean that search engine robots will then ignore the forum entirely...
URLs excluded from the XML sitemap will still be crawled because of the links pointing to them.
[edited by: martinibuster at 10:21 pm (utc) on Nov 11, 2013]
Msg#: 4622487 posted 9:41 pm on Nov 11, 2013 (gmt 0)
welcome to WebmasterWorld, Christa!
one advantage to a sitemap is that if you submit a sitemap to GWT there will be a report available showing the number of submitted urls and the number of indexed urls. you'll often see complaints about the accuracy or recency of the numbers, so take it for what it's worth.