Welcome to WebmasterWorld Guest from 54.147.44.13

Message Too Old, No Replies

Is it finally time to do away with the xml sitemap?

     
2:43 pm on Jun 29, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member sgt_kickaxe is a WebmasterWorld Top Contributor of All Time 5+ Year Member

joined:Apr 14, 2010
posts:3169
votes: 0


I've been contemplating the removal of my xml sitemap creating script for some time as it feels like its use is limited.

- Not much use to search engines if your pages are already indexed.
- No use to real visitors.
- A small performance gain can be had by removing plugins that create them.

To replace the sitemap, and to help visitors, I'm leaning towards creating an HTML sitemap instead, an archive of sorts on my site.

While I've used xml sitemaps to guide a search engine back to my content after implementing redirects or when indexing new content I don't see the value on an established and well crawled site anymore.

Question: Would removing a sitemap from a well established site cause a rankings effect of any kind?
7:55 pm on June 29, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member tedster is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:May 26, 2000
posts:37301
votes: 0


I think XML sitemaps have strong value for large sites with lots of churn. Other than that situation, it might be very worthwhile to look at the total resource required.

An HTML sitemap is a good idea even if you still have the xml sitemap. Make it more of a "guided tour" approach, rather than trying to link to every page on a mammoth site.
8:15 pm on June 29, 2011 (gmt 0)

New User

joined:June 21, 2011
posts:11
votes: 0


I think an XML sitemap definitely has its place on a site with hundreds of thousands of product pages. I feel like it can help them get indexed. I feel there is a small chance that with so many products, they will be indexed in any timely manner otherwise.

For a small website, yeah they can seem kind of pointless.
8:25 pm on June 29, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Dec 15, 2003
posts:2606
votes: 0


I am not sure that removing it will affect your rankings but it is so hard to tell now.

I know I have removed them from sites with less than 10 pages and hadn't noticed any drops but the weren't in competitive keyword markets.

I find the XML sitemap very useful when submitting new sites to be indexed, but I agree once the all your pages are indexed and there isn't a lot of content turnover I don't see a benefit.
11:30 pm on June 29, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Dec 19, 2004
posts:1939
votes: 0


I find they work fantastic in at least getting Google to understand, quickly, that there is new content at url X.

We publish the content, then update the xml feed, then 24+ hours we syndicate.
12:58 am on June 30, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:July 3, 2002
posts:18903
votes: 0


I've never used an XML sitemap.

A tight structure and sane URL hierarchy is enough on small sites.
3:11 am on June 30, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member sgt_kickaxe is a WebmasterWorld Top Contributor of All Time 5+ Year Member

joined:Apr 14, 2010
posts:3169
votes: 0


I took the plunge and removed the sitemap/plugin from a wordpress site with roughly 4000 visitors a day. I removed mention of it in robots.txt and deleted it from webmaster tools (Bing and Google).

I'd like to make other speed related improvements to the site but will wait 2 weeks between changes to get a good idea of effect so for now... tic-toc.
9:50 am on June 30, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:May 7, 2003
posts: 750
votes: 0


I do mega sites. Even on mega sites, the value of the xml sitemap is limited if your site has a crawl-able hierarchy.

Currently my XML sitemaps have every crawlable url in them. On some sites we use the priority field based on how good the page should be at ranking for keywords.

I'm thinking about trying something new. I'm thinking about only putting new pages and pages that have been updated since they were last crawled in the sitemap. With any luck this would let Googlebot focus its resources on the places that it actually needs to be crawling. I'd see boosts for QDF searches. Maybe Googlebot would even stop crawling some of my pages that rarely change multiple times a day.
10:43 am on June 30, 2011 (gmt 0)

Preferred Member

5+ Year Member

joined:Mar 29, 2007
posts:592
votes: 0


One of the use of XML sitemap of a very large site is that the sitemap section of webmaster tool gives number of URL's indexed in Google and any anomaly (say penalty to a section) might get detected early if you are looking at this number periodically.
I regularly keep track of it as site: operator give too vague numbers(at times)....
6:05 am on July 6, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member sgt_kickaxe is a WebmasterWorld Top Contributor of All Time 5+ Year Member

joined:Apr 14, 2010
posts:3169
votes: 0


7 days after removing it traffic is completely unchanged.

There is no requirement to using a sitemap so I didn't expect a loss in traffic but since it was an existing sitemap... you never know. Well, now you do.
7:10 am on July 6, 2011 (gmt 0)

Senior Member from GB 

WebmasterWorld Senior Member 5+ Year Member Top Contributors Of The Month

joined:Sept 16, 2009
posts: 991
votes: 48


One of the use of XML sitemap of a very large site is that the sitemap section of webmaster tool gives number of URL's indexed in Google

I have always found it rather daft that Google won't give you indexing data otherwise!
8:23 am on July 6, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member g1smd is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:July 3, 2002
posts:18903
votes: 0


I would have said yes until I saw the announcement about On-demand Indexing for Google CSE data recently made over at code.google.com.
9:16 am on July 6, 2011 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member tangor is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 29, 2005
posts:6130
votes: 277


Never fed the gorg or any other SE a sitemap. I've been testing their crawl capability. If they can't find all the pages I made available I put them 403 in my .htaccess as persona non grata...

Almost hit SUBMIT then changed my mind... that's a bit of humor, but is EXACTLY the reason why I've never used a sitemap.
1:25 pm on July 6, 2011 (gmt 0)

Full Member

10+ Year Member

joined:June 17, 2003
posts:208
votes: 0


how big of a site (in pages) would you say might need a sitemap?
anything over 100?
4:14 am on July 21, 2011 (gmt 0)

Senior Member

WebmasterWorld Senior Member sgt_kickaxe is a WebmasterWorld Top Contributor of All Time 5+ Year Member

joined:Apr 14, 2010
posts:3169
votes: 0


I would have said yes until I saw the announcement about on-demand indexing for Google CSE data recently made over at code.google.com.


I saw that too but it only impacts results displayed on your site, not in Google's index. The two are not different for very long so the resources required may not be worth the benefit.

The added functionality of adding an expires tag to urls in your sitemap is great but again I'd probably upload a sitemap for the purpose of removing something on an as-needed basis.

Still no sign of any rankings loss btw. Slightly increased crawl rate but not out of the norm so it's likely not attributable to the sitemap removal.
11:47 am on July 21, 2011 (gmt 0)

Junior Member

joined:May 24, 2011
posts: 58
votes: 0


Sgt_Kickaxe, thanks for bringing up this topic. I just removed my XML sitemap few moments ago. I do have an HTML one.

I run a small hobby site (forum). I'm curious to see if the removal of the XML sitemap impacts anything at all. I suspect (and hope) it will not. :-)
12:03 pm on July 21, 2011 (gmt 0)

Junior Member

joined:May 24, 2011
posts: 58
votes: 0


The following quote is from Google Webmaster Tools > Site Configuration > Sitemaps

Submit a Sitemap to tell Google about pages on your site we might not otherwise discover.


None of the pages in my XML sitemap would not otherwise be discoverable, so if "discoverability" is the only benefit of XML sitemaps, my site should not be impacted by the removal of it.