SC = Search Console, not enough title space to write it out.
Clean Up the Cruft = reference to Google's removal tool help page telling you not to use it to "Clean Up The Cruft".
SC can be quite annoying when it tells you some URL from years ago has a 301 or 404 header response when you know this because you did this and want this. There is no "shut up SC" button, only a Validate Fix button that will fail to fix this annoying problem. You can fix it and prevent future problems, though. Here's how.
- Read and re-read this page to wrap your head around what Google is really saying. Especially about making it permanent: [
support.google.com...]
- Important, do not use NOINDEX, or block with ROBOTS.txt while doing any of the below. It won't work if you do, Google needs to see this.
If the 404 error is from a URL that will never exist again, and isn't redirected - Head over to blogger.com, create a junk blog and write a blog post. "Hey G - stick your 404 error here" is fine as a title and the content. Publish it.
- On your site, create a new page on the old URL. Give it the same title and content as on your blogger post. Publish it(give it an old publish date so nobody sees it if you're on wordpress)
- Add a canonical tag to the old URL page so that it declares the blogger post as the one you want indexed
- In Search Console have Google re-crawl the url so that it sees the canonical
- Wait 4 days, Search console will tell you that the page on your site is duplicate(3 day delay in SC reporting)
- If it takes longer don't worry about it, so long as you inspect the URL on your site and see the canonical as that of the blogger page you're good.
- DELETE the page from your site, make sure it returns a 404 header response
- Use the page removal tool to remove the page URL from your site.
That's it. So long as Google NEVER sees anything but a 404 response from that old URL again it will stop telling you it exists and has errors. The problem is that although Google stopped indexing it long ago it was never truly and properly removed. Leave the Blogger page there forever. You can use the same page for all such errors in SC as long as you keep using the identical title, content and canonical when you re-create those old pages.
If the error is on an old page redirecting to a new page - Copy the content and title of the NEW page over onto the OLD page.
- Remove the redirect
- Add a canonical tag to the old page pointing at the new
- Have google re-crawl the old page to see the new canonical tag
- Wait a few days until SC reports the old page as duplicate, ideally, but it works even if Google only lists the new page as the user chosen canonical
- Delete the OLD page so that it serves a 404/410 header response, and ONLY a 404 or 410.
- Use the page removal tool to remove the OLD page URL from your site.
That's it. Google will have had a fresh look at the old URL, seen that it is duplicate to the new URL, seen that you want the new URL as the canonical, then watched the old URL vanish. In Google's eyes, as reported by SC, the page content is not gone but the URL is duplicate and all backlink juice or other age benefits, if any, will be applied to the new URL. Again, do not ever redirect the page again or allow anything but a 404 to appear or Google will once again treat it like it exists.
It doesn't matter if the page is linked to from another site at this point, Google knows what the site owner wants and the error reporting stops.
The cruft is now clean, according to SC. Who knows what search thinks though I've seen a few pages instantly perform better after cleaning the errors up. There is no 3 day wait in search results like there is search console, it's more like 24 hrs).
Now, if you read the Google URL I linked above you will have seen this under the "Permanent" section.
If you blocked the page before removing your content permanently (step 1), unblock and then reblock the page. This clears the page from the index, if it was recrawled after blocking.
Well, that's essentially what you are doing with the above, and it works.
My take is a little between the lines of that article. I think Google handles canonical tag changes far better than 301 redirects, and probably trusts them far more as well, so if you don't use methods which stop Google from seeing a page and you tell it where to find the right pages with a canonical tag the response is good, and fast. Using 301 redirects seem to trigger a 3 MONTH delay in actual search, or more, before all value is passed along. (based on old traffic vs new traffic to the same content after a move).
Hope this gives you ideas to try.
edit: The help page I linked tells you not to use the removal tool to accomplish any of the above BUT it's a part of a process that works to do just that. In the when to use this tool section it clearly says "You must take additional steps to remove the URL permanently."... and I've described them above for old errors and 301 errors in SC. I've tried it, it works like a charm. I don't look at 301's the same anymore though.
For the novice user don't try this stuff, Google takes care of it all eventually like they say. there are just some situations where an SEO needs instant results, perhaps to show a client the errors are gone, and this has worked for me.
