Server Header vs. Browser Status

Will non-dynamic custom error page confuse spiders?

[rogerd]

I've been wrestling with setting up a custom error handler on an Win2K server, and seem to be running into problems with server headers and displayed urls.

In frustration, I replaced my asp error page with a plain, old static "Ack!" type page of the same name. (The server is set up to process .htm pages as ASP.) Just for the heck of it, I checked the server headers as well as the browser status code. The server header came back with "302 Object moved", with the URL ".../error.htm?http:/www.mydomain.com/nopage.htm" where nopage.htm is the nonexistent page I requested. Using SpiderSim to check the browser status code, I got a "200 OK" status, and the URL was shown as http:/www.mydomain.com/nopage.htm. Neither was the 404 one would like in this situation.

Is this standard IIS behavior? I'm concerned about duplicate content and/or my error page being indexed every time the spider requests a page that isn't there. Do Googlebot and other spiders look at the browser status, the server header, or both, to determine the status of the requested page?

[korkus2000]

I know spiders look at the server header, but I am not sure about the browser header; although, I would think it does. I am not sure if this is normal IIS behavior, but you won't get hit for duplicate content because of the 302. Googlebot will still try that page, because it isn't a 301, but it won't associate the dead links to the new page from the 302.

[rogerd]

I've done a bit more research... There seems to be something in the IIS error transfer mechanism that messes up the ability to set the Response.Status and Location.

Here's the dilemma: If I access my custom error page directly, e.g., by typing error404.asp into the browser (with or without a query string), the following commands work fine:

Response.Status = "200 OK"
Call Response.AddHeader("Location", NewURL)

On repeated tests, I successfully changed the Response.Status to different values with no problem.

However, if I accessed the error page via an error (which the server redirects), it ignores the Response.Status and Location variables every time, returning a 302 Moved Permanently and the error page + query string location.

In short, the custom error page works properly unless it gets a referral due to a page not being found, which of course is a rather important limitation. So, is there a server setting (IIS 4.0 in this case) that avoids this apparent "forced" value for Response.Status on an error page referral? Or am I barking up the wrong tree?