homepage Welcome to WebmasterWorld Guest from 54.163.84.199
register, free tools, login, search, pro membership, help, library, announcements, recent posts, open posts,
Pubcon Platinum Sponsor 2014
Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
Forum Library, Charter, Moderators: goodroi

Sitemaps, Meta Data, and robots.txt Forum

    
How to create a sitemap for a very large site?
tpb101




msg:4679900
 4:33 pm on Jun 14, 2014 (gmt 0)

I need a sitemap that I can submit to Google Webmaster tools.

I am talking about a website, which is showing around 39,600,000 results for "site:domain-name.com".

I know that a sitemap can contain multiple sitemaps (lets say 20 or 30 etc). This is probably what I would need to do.

Do you have an idea where would I create something like that, and preferably for free?

Thank you.

 

murcy




msg:4681472
 11:24 am on Jun 20, 2014 (gmt 0)

Google webmaster doesn't allow you to create sitemap for up to 500 pages you need use paid method for that then have option to for single XML sitemap for one page.

I also create for my client website use this method.

not2easy




msg:4681520
 3:14 pm on Jun 20, 2014 (gmt 0)

For a website so large as you describe you likely have pages arranged into separate directories or folders. Usually a sitemap for each directory is created as an index for sub-sitemaps so for your site at example.com you would have sitemap-index.xml in your root directory that lists the additional sitmap-index.xml files for each folder.

There are many free and paid solutions for creating sitemaps, depending on the structure of your site. Is this a site of static html pages? A WordPress site, or Joomla? The best solution depends on how the site is built.

@murcy, it appears someone might have given you erroneous information, there is no 500 page limit and no requirement to use a paid solution to create a sitemap.

tangor




msg:4681637
 1:40 am on Jun 21, 2014 (gmt 0)

I believe the "limit" is 50,000 urls per sitemap.
The sitemap cannot exceed 10mb uncompressed
If more than 50,000 urls, then a sitemap-index to
A maximum of 1,000 sitemaps of 50,000 urls each
That would be 50,000,000 urls.

Your sitemap-index will point to where the other sitemaps are located, which can be anywhere on the site.

That said, making that kind of sitemap may not do as you wish as it is unlikely that g, or anyone else, will index all 50m urls.

Better practice is the top 1,000 to 10,000 urls (which surely have their own internal links that will get crawled) and if selected properly, should highlight all the important parts of any website.

Raj Prajapati




msg:4685572
 7:29 am on Jul 7, 2014 (gmt 0)

If you want to handle big sitmap, talk to your cms provider and get plugin or tool.

drinstech




msg:4707864
 10:12 am on Oct 11, 2014 (gmt 0)

If you are using WordPress Plugin, then you don't need to do it manually, Just install the WordPress SEO by Yoast plugin, it will automatically create a sitemap file for your blog/site. If you use others CMS or Manually developed CMS, then you can use some third party website to create sitemap file.

Global Options:
 top home search open messages active posts  
 

Home / Forums Index / Search Engines / Sitemaps, Meta Data, and robots.txt
rss feed

All trademarks and copyrights held by respective owners. Member comments are owned by the poster.
Home ¦ Free Tools ¦ Terms of Service ¦ Privacy Policy ¦ Report Problem ¦ About ¦ Library ¦ Newsletter
WebmasterWorld is a Developer Shed Community owned by Jim Boykin.
© Webmaster World 1996-2014 all rights reserved