Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

Google sitemap - do you need all forum pages?

         

lee_sufc

8:42 am on Jun 24, 2007 (gmt 0)

10+ Year Member Top Contributors Of The Month



I currently have a sitemap.xml file but at present, only include example.com/forum instead of listing every forum page. Is this OK?

tedster

7:14 am on Jun 25, 2007 (gmt 0)

WebmasterWorld Senior Member 10+ Year Member



The sitemap is only an added help to googlebot. It will still crawl links and add URLs in the normal fashion if you cannot keep up with the growth of your forum in the sitemap. No worries.

netchicken1

7:33 am on Jun 25, 2007 (gmt 0)

10+ Year Member



Get a good sitemap program, and it will index all your forums automatically.

Take care not to index the same content more than once. - I fell in the the suplimentaries with my site because of doing that, took months to get out.

Best is to use your robots file to filter all possible duplicates. For me most dups came from action= in the url (reply, quote, print, etc)and I put disallow action in the robots txt.

I bought a good program, it was worth the small amount. Online free ones *may* leave your site open to site scrapers.

Google now indexes my site thoroughly with old threads being as well hit as new ones in google. Its well well worth the time and effort in creating the sitemap.