Google is indexing my custom error pages

[adamm2]

the title explains it all... google is indexing my 401 and 403 custom error pages.

It also indexes my custom 500 error page. Why? There is no link to it.

[edited by: tedster at 4:37 pm (utc) on Mar. 30, 2009]

[tedster]

Hello adamm2, and welcome to the forums.

The reason this usually happens is that your server is not returning a 404 status code in the http header - that's the information the server sends to the user agent before it sends the page's visible content.

This has been called a "soft 404" and the fix is just to return the actual error status code in the http header. You can use Firefox with the LiveHTTPHeaders add-on to inspect all the http header information going back and forth between a user agent and your server. This will help you to be sure when you have it right.

[LifeinAsia]

As an added measure, you can also add the following line in the HEADER of your customer error pages:
<meta name="Robots" content="NOINDEX">

[adamm2]

@tedster:
It IS returning the proper errors codes as the error pages show up in Google's Webmaster Tools under the Web Crawl Diagnostics in the "HTTP Errors" section. and every error page that shows up shows up saying it has an error.

For example, in the Diagnostics, for "mydomain.com/error/401" it shows "401/407 authentication error" under the detail column... same for all the other error pages.

@LifeinAsia: I think your suggestion would work