Welcome to WebmasterWorld Guest from 18.104.22.168
Forum Moderators: incrediBILL
I have modified my 404 not ofund page to return a custom erro page (1.5k in size).
My question is now: will all those spiders/bots realise this page is an error page, or will they happily index it as well?
I know about the robot exclusion std. and the possibility I have with the robots meta tag.
It runs from the command line, and outputs results in the telnet window, and I've now weeded out all my in-site 404s. Right now the only 404s I'm getting are people (like me) mi-typing URLs.
checklinks info & download: