Forum Moderators: Robert Charlton & goodroi
Which would have been better, from a search engine perspective, a 404 or 410 for the pages that are gone and will not be back or the redirect to a similar page?
My thoughts at the time were, the redirect, so that our customers would at least hit a similar page on our site instead of just encontering a 404 error and never reaching our site when coming in from the search engine.
This will lead to... yes, duplicate content, it's logic after all.
The best way to go would be to put a 404 or 410 with some kind of site map on it, where you can point the users to the home page or other pages of interest.
fourchette
If you will be removing the page and there is no replacement, then you should consider a custom 404 error page.
If you do use the custom error page remember:
1. noindex,nofollow the page.
2. you need to put the code in your .htaccess file yourself. (DO NOT use your server interface or account management software) If this is not set correctly, you will end up serving a 302 instead of a 404.
3. The following is the correct syntax:
ErrorDocument 404 /the/path/to/your/file.html
(DO NOT use http:// when setting the location.)
Hope this helps.
Justin
I now redirect them to a page similar to my sitemap with links to the most common pages on my site, but this page uses the robots meta "noindex" and it also returns a proper 404 HTML code. In this way I serve both the SE's and the visitors. The search engines see a 404 response code and a "noindex" tag which prevents duplicate content and the visitor gets a page that brings him to the needed information in only one or two steps. Only when there is an almost identical replacement page, I use a 301 redirect instead of this 404 page.
I see how logically it would create a duplicate content issue, but this hasn’t been my experience. What I’ve seen so far is that the old non-existent page just gets dropped from the index.
That is what happens when the redirect is a 302 redirect. You get the same content indexed under any URL that serves a 302 redirect and then takes you to that content. Make sure that the redirect is a 301 redirect and you avoid the problem.
The 301 redirect AVOIDS duplicate content issues. Only the single URL that returns "200 OK" is the one to be indexed. All of the other URLs are noted as being (301) redirects but nothing is indexed against those URLs, nor should it be.
Had you used a 302 redirect you would have been in big trouble.
Although the 301 redirect worked for you, it isn't the best solution. It isn't a good idea to redirect lots of sub-page URLs to the root index. Instead you should serve a custom error page, one that contains an error message and some basic site navigation. Served with a status of "404", Google will quickly drop those old URLs while allowing all visitors to still access the site.
Custom 404's are going to be my project this weekend ;)
Make sure you check that the status (for URLs that no longer exist) really is "404" using WebBug or similar.
That is what happens when the redirect is a 302 redirect. You get the same content indexed under any URL that serves a 302 redirect and then takes you to that content. Make sure that the redirect is a 301 redirect and you avoid the problem.>
I'm not sure what it is - it is an off the shelf shopping cart and the redirect is in the programming somewhere.
If you look up the headers of the redirected pages, it returns a 200 ok (which also means that you can't remove them from Google because it thinks they are valid pages). The company who created the shop won't provide support for changes in php coding, so I haven't a clue how to change it to redirect anywhere else. The only thing I can do is add a redirect to a 404 page to each specific dead product or section page, but I'm not sure how to do this even! A usual redirect in .htaccess?
For your screwed up shopping cart (the designers of these things have no clue what they are doing) you need to detect when the redirect is occurring and add <meta name="robots" content="noindex"> to the <head> section of the page so that at least none of those fake URLs can get listed. The other advantage is that all of the existing duff listings will be dropped too.
I suggest visiting the PHP forum... somewhere in the script there is a connection to the database --- It is usually best to set your headers after you connect and know if there are any results for what was requested.
To correct the 200/404 problem you can add a line something like this:
if(!$results OR $results==='') {
header("HTTP/1.0 404 Not Found");
Other stuff you would like to have on your error page here --- maybe open an html (custom 404) page. Be sure to follow the suggestions above and 'noindex' anything you display.
exit;
}
** The first line says if $results has not been set, or there is no information in $results serve a 404 status to the browser/user-agent.
If you would like to redirect for some reason, you will need something like this:
header ("HTTP/1.0 301 Moved Permanently");
header("Location: $URL");
** This one will set a 301 on your redirects.
Hope this gives you some ideas.
Justin
EDITED: Decided to translate what was in my head to readable English.
