Welcome to WebmasterWorld Guest from

Forum Moderators: goodroi

Message Too Old, No Replies

How to create a sitemap for a very large site?

4:33 pm on Jun 14, 2014 (gmt 0)

New User

joined:Apr 23, 2014
votes: 0

I need a sitemap that I can submit to Google Webmaster tools.

I am talking about a website, which is showing around 39,600,000 results for "site:domain-name.com".

I know that a sitemap can contain multiple sitemaps (lets say 20 or 30 etc). This is probably what I would need to do.

Do you have an idea where would I create something like that, and preferably for free?

Thank you.
11:24 am on June 20, 2014 (gmt 0)

New User

joined:Apr 26, 2014
votes: 0

Google webmaster doesn't allow you to create sitemap for up to 500 pages you need use paid method for that then have option to for single XML sitemap for one page.

I also create for my client website use this method.
3:14 pm on June 20, 2014 (gmt 0)

Administrator from US 

WebmasterWorld Administrator not2easy is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Dec 27, 2006
votes: 182

For a website so large as you describe you likely have pages arranged into separate directories or folders. Usually a sitemap for each directory is created as an index for sub-sitemaps so for your site at example.com you would have sitemap-index.xml in your root directory that lists the additional sitmap-index.xml files for each folder.

There are many free and paid solutions for creating sitemaps, depending on the structure of your site. Is this a site of static html pages? A WordPress site, or Joomla? The best solution depends on how the site is built.

@murcy, it appears someone might have given you erroneous information, there is no 500 page limit and no requirement to use a paid solution to create a sitemap.
1:40 am on June 21, 2014 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member tangor is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 29, 2005
votes: 547

I believe the "limit" is 50,000 urls per sitemap.
The sitemap cannot exceed 10mb uncompressed
If more than 50,000 urls, then a sitemap-index to
A maximum of 1,000 sitemaps of 50,000 urls each
That would be 50,000,000 urls.

Your sitemap-index will point to where the other sitemaps are located, which can be anywhere on the site.

That said, making that kind of sitemap may not do as you wish as it is unlikely that g, or anyone else, will index all 50m urls.

Better practice is the top 1,000 to 10,000 urls (which surely have their own internal links that will get crawled) and if selected properly, should highlight all the important parts of any website.
7:29 am on July 7, 2014 (gmt 0)

New User

5+ Year Member

joined:Mar 5, 2011
votes: 0

If you want to handle big sitmap, talk to your cms provider and get plugin or tool.
10:12 am on Oct 11, 2014 (gmt 0)

New User from BD 

joined:Oct 11, 2014
votes: 0

If you are using WordPress Plugin, then you don't need to do it manually, Just install the WordPress SEO by Yoast plugin, it will automatically create a sitemap file for your blog/site. If you use others CMS or Manually developed CMS, then you can use some third party website to create sitemap file.
8:13 am on Dec 5, 2014 (gmt 0)

New User

joined:Nov 27, 2014
votes: 0

Generating XML Sitemap for larger website is not a difficult task as mentioned. Nowadays, there are many third party tools available which can help to generate Sitemap for 'n' number of URLs.

Also agree with 'drinstech', if your website is present on WordPress you can generate the sitemap using Plugin provided - WordPress SEO by Yoast.

Let me know if you need any help to generate the Sitemap. You can share the website URL as it will help to assist you further.
6:50 am on Dec 8, 2014 (gmt 0)

New User

joined:Nov 24, 2014
votes: 0

No need to create a single sitemap. you can create multiple sitemaps. Create one sitemap for each category or services. You can read online about creating multiple sitemap and naming convention.
11:13 am on Jan 21, 2015 (gmt 0)

New User from IN 

joined:Jan 21, 2015
votes: 0

If you want to create a sitemap for thousands of pages that you need to make multiple sitemap or you can use paid sitemap for unlimited pages.
11:26 am on Jan 21, 2015 (gmt 0)

New User

joined:Nov 11, 2014
posts: 1
votes: 0

I think your question is somewhat confuse, but freely you can create upto 500 sitemaps for each subdomains, no need to create more than one for each website, if you have subdomains you can create and submit

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members