Welcome to WebmasterWorld Guest from 50.16.112.199

Forum Moderators: goodroi

to map or not to map

   
10:30 pm on Apr 28, 2013 (gmt 0)

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time Top Contributors Of The Month



One of those fuzzy philosophical questions:

Here in this bright and beautiful year of our lord 2013, when-if-ever is a sitemap necessary? I'm asking here about xml or equivalent sitemaps aimed at search engines, not at visible sitemaps for the benefit of humans.

The book says sitemaps are to help the search engine find pages they might otherwise miss. When does this happen?

Obviously any page linked by <a href... is out. Robots can find those just fine. As I understand it, a "nofollow" tag makes no difference; that just applies to indexing, not to physically locating a file. Links created by javascript used to be an issue, but the major search engines will happily execute scripts so that's another non-starter. Directories requiring logins might be a problem, but there's always the "satisfy any" directive-- and why are you indexing members-only pages anyway?

For comparison purposes: I used to have a pretty comprehensive sitemap. About a year back I said the heck with it and cut it back to only directory-index pages-- either files whose actual name* is index.html, or named pages performing the same function. So even if the server explodes and all sitewide navigation footers vanish into the ether, the search engine can still find everything.


* That is, ahem, the real name of the physical file. Not the URL.
 

Featured Threads

My Threads

Hot Threads This Week

Hot Threads This Month