Custom error page?

A questino about custom error pages?

[skirril]

Hello,

I have modified my 404 not ofund page to return a custom erro page (1.5k in size).

My question is now: will all those spiders/bots realise this page is an error page, or will they happily index it as well?

I know about the robot exclusion std. and the possibility I have with the robots meta tag.

Skirril

[sugarkane]

If you're doing it through htaccess, the server will return a 404 status along with the error page and the spiders won't (shouldn't!) index it.

[tedster]

That's the way things have been working for me for quite some time -- thank goodness, since on some sites the custom error page is a copy of the index page. Wouldn't want to get that dropped as a duplicate!

[rcjordan]

While on the topic, I added javascript to write the referrer to a text block on a form on the 404 custom page. All the user has to do is click the "submit" button and it'll send the url of the page where the bad link is coded. You'll be surprised/shocked at how many you'll get.

[mivox]

I found a free script: Checklinks 1.0.1 It's very customizavle, as far as what you want it to check, report, ignore, etc. (I have it set to ignore my javascript links... I check those by hand whenever I change a page or create a new one...)

It runs from the command line, and outputs results in the telnet window, and I've now weeded out all my in-site 404s. Right now the only 404s I'm getting are people (like me) mi-typing URLs.

checklinks info & download:
[jmarshall.com...]