I want to create a simple site map for my website. It is all just html pages so it should be fairly easy. I think I've confused myself with the navigation myself......lol Each page has the same hierarchal navigation as well as links to other pages and external links.
Should I include all the external links in the site map? Also I'm not sure if I should repeat all the hierarchal links in each section as they are present on each page. Or if I do that then I could be just repeating myself.
I've started a site map in a word doc as I rough guide and it looks too big if I include the hierarchal links.
The default site map can allow the crawler to entire site. If you want to restrict the spider to view some of your web page, you can use the allow and disallow comments at the site map. Site maps are the links used to guide the crawler.