Welcome to WebmasterWorld Guest from 3.84.130.252

Forum Moderators: Robert Charlton & goodroi

Message Too Old, No Replies

Increase in “404” pages on my site – should we make 301 redirection?

     
6:51 am on May 15, 2018 (gmt 0)

Preferred Member

5+ Year Member

joined:June 10, 2011
posts: 537
votes: 0


Hello,
We received an email message from Google Search Console Team that Googlebot for smartphones (and desktop) found many 404 Urls on our site. Actually hundreds ..
Digging into the list of URLs in GWT, it was obvious for us that these URLs definitely do not exist.
Most of them showing the first part of an existing URL - we saw that Spammy websites linking to wrongful URL.
For example - www.my-site.ABCD instead of www.my-site.ABCDEF.html

The question is – would it be wise to 301 redirect all these URLs to the correct ones? (for getting some backlinks, making sure to redirect to the most appropriate related pages)
8:12 am on May 15, 2018 (gmt 0)

Administrator

WebmasterWorld Administrator phranque is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Aug 10, 2004
posts:11684
votes: 205


check your logs to see what other traffic besides googlebot is requesting these urls
4:56 pm on May 15, 2018 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:15640
votes: 796


would it be wise to 301 redirect all these URLs to the correct ones?
If and only if
(a) the wrong URLs are getting requested by humans
and
(b) you can identify the real page behind the erroneous URL (as in the recent Apache thread involving requests for /realdirectory/.../realsubdirectory/realpage.htm, apparently thanks to careless cut-and-paste work).

If the page has no existence outside G###'s fevered imagination, a 404 is the appropriate response.
5:36 pm on May 15, 2018 (gmt 0)

Preferred Member

Top Contributors Of The Month

joined:Mar 25, 2018
posts:500
votes: 101


if it's coming from spammy web sites, I doubt that Google gives a lot of value to these links. Personally, I would 404 them.
6:14 pm on May 19, 2018 (gmt 0)

New User

joined:Feb 25, 2018
posts:10
votes: 5


I redirect all my 404 pages to search pages.
I've got no idea if it's the best method but it suits me and the errors have gone in gsc.
6:38 pm on May 19, 2018 (gmt 0)

Senior Member

WebmasterWorld Senior Member Top Contributors Of The Month

joined:Apr 1, 2016
posts:2552
votes: 722


@meagain121
I redirect all my 404 pages to search pages.

That is a very bad idea, that results in what Google calls a soft-404, and is a situation that should be avoided. A request for a page that does not exist should return a 404 "Does not exist" response. The existence of 404's has no impact on ranking.
10:42 pm on May 19, 2018 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:15640
votes: 796


I redirect all my 404 pages to search pages.
Why not put a search on your 404 page? Then everyone is happy.
10:42 pm on May 19, 2018 (gmt 0)

Moderator This Forum from US 

WebmasterWorld Administrator robert_charlton is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Nov 11, 2000
posts:12295
votes: 389


Zivush, this is from an extensive Jan 11, 2013 Google+ post John Mueller made about how Google looks at these, and it may well be helpful. I'm quoting the first 2 of 7 observations John made, and I strongly recommend reading and absorbing the whole post...

John Mueller > Public
https://plus.google.com/+JohnMueller/posts/RMjFPCSs5fm [plus.google.com]

HELP! MY SITE HAS 939 CRAWL ERRORS!1

I see this kind of question several times a week; you're not alone - many websites have crawl errors.

1) 404 errors on invalid URLs do not harm your site's indexing or ranking in any way. It doesn't matter if there are 100 or 10 million, they won't harm your site's ranking. [googlewebmastercentral.blogspot.ch...]

2) In some cases, crawl errors may come from a legitimate structural issue within your website or CMS. How you tell? Double-check the origin of the crawl error. If there's a broken link on your site, in your page's static HTML, then that's always worth fixing. (thanks +Martino Mosna)
And five more observations worth reading, with links to longer posts worth checking out too.

11:16 pm on May 19, 2018 (gmt 0)

Administrator

WebmasterWorld Administrator phranque is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Aug 10, 2004
posts:11684
votes: 205


I redirect all my 404 pages to search pages.
I've got no idea if it's the best method but it suits me and the errors have gone in gsc.

those urls may eventually show up in GSC's "soft 404s" report...
5:07 am on May 20, 2018 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:15640
votes: 796


Come to think of it: the errors haven't actually gone, have they. They've simply relocated to the “soft 404” tab.
6:21 pm on May 23, 2018 (gmt 0)

New User

joined:Feb 25, 2018
posts:10
votes: 5


Thanks for the feedback.

My 404 errors were from old pages, years ago (they would just reappear as 404s in GSC if I deleted them, even though they weren't in my sitemap).

The errors have now gone and no 404s or soft 404s came back.

I added the redirects as a 301. FYI like this in my wp header:

if(is_404()){ //redirect 404 to search
header("HTTP/1.1 301 Permanent Redirect");
header( 'Location: https://www.mydomain.com?s='.urlencode(preg_replace('/[^A-Za-z0-9\s]/',' ',$_SERVER['REQUEST_URI'])));
die;
}


Do you see any issues?
8:06 pm on May 23, 2018 (gmt 0)

Senior Member from US 

WebmasterWorld Senior Member lucy24 is a WebmasterWorld Top Contributor of All Time 5+ Year Member Top Contributors Of The Month

joined:Apr 9, 2011
posts:15640
votes: 796


Do you see any issues?
Yes, but we've already been over that.

I assume you've got a solid reason for deciding, based on your individual site's configuration, that a php redirect is less work for the server than an -f and -d test (assuming Apache).

I hope your URLs don't actually contain spaces.
8:48 pm on May 23, 2018 (gmt 0)

Administrator

WebmasterWorld Administrator phranque is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Aug 10, 2004
posts:11684
votes: 205


Do you see any issues?

if a request for https://www.example.com/arbitrary_string/ would normally cause a 404, your script will redirect this request to https://www.example.com?s=arbitrary_string/

do i understand you correctly?

if the response for https://www.example.com?s=arbitrary_string/ is no different from the response for https://www.example.com/ then you have a soft 404 problem by definition.
googlebot just hasn't found it or GSC hasn't shown it to you yet...
11:43 pm on May 23, 2018 (gmt 0)

New User

joined:Feb 25, 2018
posts:10
votes: 5


I hope your URLs don't actually contain spaces.

I use a str_replace later :)
if a request for https://www.example.com/arbitrary_string/ would normally cause a 404, your script will redirect this request to https://www.example.com?s=arbitrary_string/

Yes, https://www.example.com?s=arbitrary_string/ returns 200 ok search page (different from https://www.example.com/).
It works for me, might not for others - just putting it out there.
11:50 pm on May 23, 2018 (gmt 0)

Administrator

WebmasterWorld Administrator phranque is a WebmasterWorld Top Contributor of All Time 10+ Year Member Top Contributors Of The Month

joined:Aug 10, 2004
posts:11684
votes: 205


https://www.example.com?s=arbitrary_string/ returns 200 ok search page (different from https://www.example.com/)

are the responses for https://www.example.com?s=arbitrary_string/ and https://www.example.com?s=another_arbitrary_string/ different?
how different?
are these (search page) urls meta robots noindexed?
 

Join The Conversation

Moderators and Top Contributors

Hot Threads This Week

Featured Threads

Free SEO Tools

Hire Expert Members