How many 404's on a site before a penalty?

[Bewenched]

We have a very large ecommerce site and brands are constantly discontinuing items and coming out with new ones. Right now when an item is permanently discontinued we remove the product from our system and return a 404 if someone goes to that product page.

When I log into WMT I've always gotten alot of 404 notifications.
I still see 404 notifications for products that haven't been on our site for years.

My question is: Can a site have too many 404's and get a penalty.
If so, then what is the best way to handle products that are discontinued?

Should we:
1) Leave them in the system and return a 200?
2) 301 redirect somewhere else?
3) Continue serving 404's? (not soft ones)
$) Is there a better option?

[tedster]

There are two kinds of 404 problems - those that come from your own internal links and those that come from other site's links to you. Neither one causes a true penalty, but too many internal 404s can certainly cause ranking troubles. I assume that when you remove a product, that also removes any internal links that pointed to it - if not, then make sure that happens.

Other sites that link to 404 URLs on your domain will get reported to you in Webmaster Tools as an "FYI". You may well want to accommodate those incorrect backlinks with a 301 or something like that, so you can get the link juice to flow. But if you ignore them, there is no negative against your site.

[jdMorgan]

The proper response for an intentionally-removed resource is a 410-Gone, unless you have a direct replacement resource to use as a target URL for a 301-Moved Permanently redirect. Historically, Google has treated a 410-Gone response as equivalent to a 404-Not Found, but now claims to "handle it" -- perhaps it will help in getting the old URLs removed from their search results sooner.

The semantics of 404-Not Found are "the resource was not found for an unknown reason" and therefore, Google has no way to know that the situation is not temporary; the resource might very well return after a broken database is re-built, for example. The 410 response removes this ambiguity, and is considered permanent.

The 410-Gone also clarifies things for users should they request a removed page -- say through an old bookmark or a link found on a third-party site.

In a case like this (on an e-commerce site), the 410-Gone page should clearly state that the resource has been intentionally removed and provide links to similar products, the product's category page, your site-search facility, your HTML (human-readable) sitemap page, or -as the very last and poorest choice- your home page. Ideally, you'd want to offer all of the above, and in that order. In all cases, these should be clickable links on a custom 410 error page which returns a 410-Gone server status code -- Do not 301-redirect to any page except a direct-replacement page from the URL of the removed page.

I'd suggest an extension of your database to include replacement URLs for products which have an acceptable direct substitute. If this field is empty, your script(s) can generate a 410-Gone response and then build the appropriate links for the error page by examining the category of the requested product. In no case should the product's database record be simply deleted. Among other things, it might be useful to be able to quickly ascertain that yes, we used to carry that, but it was discontinued and replaced by product #1234567. That might come in handy to verify warranty claims and to handle the case where a warranty replacement is called for...

When in doubt about the "correct" way to handle various missing-resource, server-outage, and redirect issues, a quick review of the "Response" section of the HTTP/1.1 protocol specification is often useful. This document describes "how the Web is supposed to work," and "getting it right" can save an awful lot of trouble later.

Jim

[Bewenched]

Yes, when a product is deleted all internal dynamic linking is removed immediately. 99% of the site is dynamic and these 404's in WMT are so

But what I'm seeing is that google goes to say a category page then to a product page and maybe a few days later that product is deleted.
Google keeps reporting that product page as being linked from the category page.... but they aren't .. I have notifications of products/pages that were removed months and months ago or even years, saw one today in the list that had not been in our system since 2007

We can turn off a product and in a field put in the replacement number and set it to superceded and customers and bots get 301 redirect to the new item.

When a 404 occurs, the page they are displayed is human readable, with search suggestions, a search box and our main menu displayed.

The problem with keeping discontinued items in the database is that we get tons of phone calls asking if we can get it, even when it says DISCONTINUED, NO LONGER AVAILABLE.
I mean as if by magic the customer service people can fabricate it for them. It was causing tons of phone calls that rarely if ever led to real sales.

I'll look more into the 410 and repercussions it could have in the future if by chance a brand brings back that productcode