Questions about 404 Redirects

All the talk about 404, 302, 301 redirects is confusing!

[abrown]

I'm a designer who's new to this fabulous world of web development. This site has been a lifesaver!

Here's today's problem:

I have inherited an ASP site that uses a custom 404 page to handle redirects for files that no longer exist. The redirects have always worked fine, but now that I've been learning about SEO, I want to make sure that all of my redirects are crawler-friendly.

I've heard this 301 thing mentioned a lot in the same context. Other than the fact that it's a permanent redirect as opposed to a temporary 302, I'm not quite sure what it is, or how to do it.

Is what I'm doing with the 404 page enough to make the crawlers happy? The reason I'm stressing is because I went into one of those "HTTP Header Viewer" tools and entered one of my redirected URLs. Although I wasn't quite sure what I was looking for, I noticed that a 302 was returned, and not a 301. No clue why. Is this normal?

How do I make my 404 redirects give the proper 301 code to the crawlers? I don't see anything in my ASP code that does this.

Am I making sense? Please excuse me if I'm not using proper terms, etc.

[abrown]

Wow. I just discovered something else. Looks like the robots.txt file is not letting the crawlers access the custom 404.asp page. Could this be a problem?

[macrost]

abrown,
Welcome to WebmasterWorld! I commonly use custom 404 pages for url rewrites. In the code of your 404 page, you should see Response.Status = "302 Found" if the header checker is seeing a 302 coming back. You need to change the 302 to "301 Moved Permanently" i.e.
<%
Response.Status = "301 Moved Permanently"
Response.AddHeader "Location", "http://www.newdomain.com/"
Response.End
%>

From this old thread. [webmasterworld.com]

HTH
<added>As to the block in robots.txt, there really isn't a reason for it to be there unless the last person somehow had a link to it. If I were you, just keep it there.</added>

[Dreamquick]

It's also worth noting that IIS in the default configuration can sometimes serve a 302 in order to redirect the browser to the 404 script which will eventually give them the 301.

- Tony

[abrown]

Thanks for your tips!

Macrost, I saw your same advice in some other threads, but nowhere in my script is there anything similar to your code. No Response.Status, etc.

All of my redirects appear in a long list like this:

<%
...
dim arrRedirects(48)
arrRedirects(0) = Array("/blah/aa.asp","/bloh/bb.asp")
arrRedirects(1) = Array("/bluh/cc.html","/blah/dd.asp")
arrRedirects(1) = Array("/bloh/dd.html","/bluh/ee.asp")
...
%>

Being new to ASP, this means nothing to me! I just know that the first one is the obsolete one, and the second one is the redirect.

I'm thinking that what's going on is something like what Tony suggests...that it's somewhere in the IIS configs. If so, could it be as simple as chaging those cofigs from 302 to 301?

Otherwise, Macrost, could there be a way to implement your code to "coexist" with what I already have?

:-) Thanks!

[abrown]

Oops and one more thing. The wasn't in the robots.txt file, it was in the meta tag of the 404.asp file. Macrost, you say to keep it, just in case there was a good reason for it. But will this mean that the crawlers can't get to my 404 page to see the redirects?

[macrost]

abrown,
The spiders will follow your old links and the 404 page will process that spider and direct accordingly. If a spider directly hits the 404 page, it could cause some errors.
Sticky me the 404 page and I'll have a peek at it.

[abrown]

Cool.

You got my first sticky ever! BTW, don't worry about the include...I can't see anything relevant in it. It's all for our site forms, etc.

Thanks so much!

[mattglet]

You could just do this:

<%
Response.Status = "301 Moved Permanently"
Response.AddHeader "Location", strRedirectURL
Response.End
%>

strRedirectURL would consist of your array value, or whatever value you are using in your response.redirect.

Should be very easy to implement.