sitemap and broken pages.

[jediviper]

As we all know, the sitemap.xml should contain no broken pages.
But happens with sitemaps that are created automatically by the system? There are thousands of links included there and I can't think any manual way to update the sitemap regularly.

So what is the standard for companies with huge sitemaps? How do they avoid the broken links from being added there or if they are already in the sitemap, how do they remove them (manually since like a very time-consuming task)?

[tangor]

One needs to adhere to the ordinary limits of site maps ... or break them up as indicated by the specs.

Created automatically how?

[not2easy]

Start with

created automatically by the system

to fix the problem. If part of the system is not doing what it needs to do, it may be time to work on changing that part of the system.

As tangor mentions, "how" the sitemap is being created is the key to getting useful suggestions here.

[jediviper]

Welcome to the Matrix..
This is what comes in my mind when they ask me why or how something is happening regarding our sites and if it's related to purely IT topics. There are not many things that I can do there as usually I hit a wall when I am trying to change things around the IT department.

But I would like more to get a good idea of what is happening with other medium to large companies, because if I have a good solution to propose, then maybe I can make some heads turn.

[not2easy]

"The System" remains a mystery here. Things that might be suggested for a site built with a custom CMS in perl vs. a custom PHP platform vs. static html vs. something like WordPress are not interchangeable. These are all platforms that might be used by medium to large companies. If you could explain what the platform in question is, then people can make valid suggestions. Without that it is a guessing game.

[lucy24]

Obligatory boilerplate reminder: You don’t need a sitemap at all. If your most important pages are not already accessible by the most easily identified links from the front page, there is a problem with the site itself.

[tangor]

Second @lucy24 ... I have never used sitemaps at all. Ever.

Always seemed like duplicate effort and distracting from the real effort: making the site optimal to all indexing.

That, and an understanding that g (or others) will display what THEY think is important, not what YOU think is important.

[tangor]

Search engines have their own product ... their serps.

[jediviper]

About the reason to use a sitemap or not.
Well maybe for websites with a few dozen pages is not required, but I believe in our case and for other websites with thousands of links it's better to have a sitemap as an aid for Google to discover and index all links.
And u can't have all these links connected to the homepage.

[tangor]

And u can't have all these links connected to the homepage.

Nor should you, but any interior page should be linked to other pages that do link back ... one, two, three or more... Else they aren't connected to the site at all, other than having a domain in common.

I spend my time with internal linking, not creating a list of files/pages where a broken link might be missed.

[tangor]

Sitemaps, however, do have useful functions for some, so use them if they make sense.