Increase in “404” pages on my site – should we make 301 redirection?

[Zivush]

Hello,
We received an email message from Google Search Console Team that Googlebot for smartphones (and desktop) found many 404 Urls on our site. Actually hundreds ..
Digging into the list of URLs in GWT, it was obvious for us that these URLs definitely do not exist.
Most of them showing the first part of an existing URL - we saw that Spammy websites linking to wrongful URL.
For example - www.my-site.ABCD instead of www.my-site.ABCDEF.html

The question is – would it be wise to 301 redirect all these URLs to the correct ones? (for getting some backlinks, making sure to redirect to the most appropriate related pages)

[phranque]

check your logs to see what other traffic besides googlebot is requesting these urls

[lucy24]

would it be wise to 301 redirect all these URLs to the correct ones?

If and only if
(a) the wrong URLs are getting requested by humans
and
(b) you can identify the real page behind the erroneous URL (as in the recent Apache thread involving requests for /realdirectory/.../realsubdirectory/realpage.htm, apparently thanks to careless cut-and-paste work).

If the page has no existence outside G###'s fevered imagination, a 404 is the appropriate response.

[Travis]

if it's coming from spammy web sites, I doubt that Google gives a lot of value to these links. Personally, I would 404 them.

[meagain121]

I redirect all my 404 pages to search pages.
I've got no idea if it's the best method but it suits me and the errors have gone in gsc.

[NickMNS]

@meagain121

I redirect all my 404 pages to search pages.

That is a very bad idea, that results in what Google calls a soft-404, and is a situation that should be avoided. A request for a page that does not exist should return a 404 "Does not exist" response. The existence of 404's has no impact on ranking.

[lucy24]

I redirect all my 404 pages to search pages.

Why not put a search on your 404 page? Then everyone is happy.

[Robert Charlton]

Zivush, this is from an extensive Jan 11, 2013 Google+ post John Mueller made about how Google looks at these, and it may well be helpful. I'm quoting the first 2 of 7 observations John made, and I strongly recommend reading and absorbing the whole post...

John Mueller > Public
https://plus.google.com/+JohnMueller/posts/RMjFPCSs5fm [plus.google.com]

HELP! MY SITE HAS 939 CRAWL ERRORS!1

I see this kind of question several times a week; you're not alone - many websites have crawl errors.

1) 404 errors on invalid URLs do not harm your site's indexing or ranking in any way. It doesn't matter if there are 100 or 10 million, they won't harm your site's ranking. [googlewebmastercentral.blogspot.ch...]

2) In some cases, crawl errors may come from a legitimate structural issue within your website or CMS. How you tell? Double-check the origin of the crawl error. If there's a broken link on your site, in your page's static HTML, then that's always worth fixing. (thanks +Martino Mosna)

And five more observations worth reading, with links to longer posts worth checking out too.

[phranque]

I redirect all my 404 pages to search pages.
I've got no idea if it's the best method but it suits me and the errors have gone in gsc.

those urls may eventually show up in GSC's "soft 404s" report...

[lucy24]

Come to think of it: the errors haven't actually gone, have they. They've simply relocated to the “soft 404” tab.

[meagain121]

Thanks for the feedback.

My 404 errors were from old pages, years ago (they would just reappear as 404s in GSC if I deleted them, even though they weren't in my sitemap).

The errors have now gone and no 404s or soft 404s came back.

I added the redirects as a 301. FYI like this in my wp header:

if(is_404()){ //redirect 404 to search
header("HTTP/1.1 301 Permanent Redirect");
header( 'Location: https://www.mydomain.com?s='.urlencode(preg_replace('/[^A-Za-z0-9\s]/',' ',$_SERVER['REQUEST_URI'])));
die;
}

Do you see any issues?

[lucy24]

Do you see any issues?

Yes, but we've already been over that.

I assume you've got a solid reason for deciding, based on your individual site's configuration, that a php redirect is less work for the server than an -f and -d test (assuming Apache).

I hope your URLs don't actually contain spaces.

[phranque]

Do you see any issues?

if a request for https://www.example.com/arbitrary_string/ would normally cause a 404, your script will redirect this request to https://www.example.com?s=arbitrary_string/

do i understand you correctly?

if the response for https://www.example.com?s=arbitrary_string/ is no different from the response for https://www.example.com/ then you have a soft 404 problem by definition.
googlebot just hasn't found it or GSC hasn't shown it to you yet...

[meagain121]

I hope your URLs don't actually contain spaces.

I use a str_replace later :)

if a request for https://www.example.com/arbitrary_string/ would normally cause a 404, your script will redirect this request to https://www.example.com?s=arbitrary_string/

Yes, https://www.example.com?s=arbitrary_string/ returns 200 ok search page (different from https://www.example.com/).
It works for me, might not for others - just putting it out there.

[phranque]

https://www.example.com?s=arbitrary_string/ returns 200 ok search page (different from https://www.example.com/)

are the responses for https://www.example.com?s=arbitrary_string/ and https://www.example.com?s=another_arbitrary_string/ different?
how different?
are these (search page) urls meta robots noindexed?