Is the site using a CMS such as WordPress? Are each of the individual URLs showing different content or are some of the 'pages' different means to find the same information? Are all pages in the same language?
You would want to create an index to list the sitemaps separated to the size limits of 50,000 URLs and you can create multiple indexes as needed. The sitemaps can be in compressed format such as gzip.
Yahoo, Google and MicroSoft set the format so their crawlers could all use one (xml) syntax. Sitemaps.org is where the basics are available, including how to sort and create your sitemaps: [sitemaps.org...]
lucy24
4:12 pm on Jul 5, 2023 (gmt 0)
Obligatory reminder: If humans can find all your pages, so can search engines. A site map is never an absolute necessity.
If you fear that some pages are hard to find, think about making them more reachable for people already on the site. For me, few things are more infuriating than arriving at a page via a search engine, spending some time puttering around the site ... and then being utterly unable to find the original page again from within its own site.
Sanjaya87
5:05 pm on Jul 5, 2023 (gmt 0)
Thank you
tangor
4:06 am on Aug 12, 2023 (gmt 0)
Reminder: Just because you have a huge sitemap does NOT mean g will crawl all of it immediately. They have their crawl budget and will get through it at their own pace (making decisions about your site along the way) and that's the best it will do.
On the other hand, a much smaller targeted sitemap to "push" desired high value/user experience URLS to g will give better results.
That said, I have never used a sitemap and never had any problems with getting serp love.
YMMV
Kendo
12:28 am on Aug 15, 2023 (gmt 0)
There are some online sitemap generators. But if their output is limited I would start with something like Xenu to get a list of all links found from your homepage.
Then I would edit that file to remove unwanted info and add the necessary code before and after your links using search-n-replace.
tangor
11:19 am on Sep 5, 2023 (gmt 0)
A 4 million url website is at least 81 sitemaps. 80 for the urls and one index sitemap to list the others. :)
(standards: 50,000 urls per sitemap, 50,000 sitemaps total, none over 50mb in size, 2.5 billion urls max)
Largest site I ever worked for a client was just shy of 900,000. They wanted a "sitemap" as well. Started working on that and then discovered that nearly half the website was not FULLY LINKED (broken navigation) and when all those errors were fixed the sitemap was no longer needed. BTW ... Xenu, at that time, gave up at about 400,000 (unable to complete). I do not know of anything that will do a sitemap of 4 million and break it down into the required sitemap specifics.
