Welcome to WebmasterWorld Guest from

Forum Moderators: Ocean10000 & incrediBILL & keyplyr

Message Too Old, No Replies

file not found



1:09 pm on Apr 17, 2001 (gmt 0)

Inactive Member
Account Expired


We had performed a major upgrade to our website a couple of months ago. This upgrade had caused most all of the previous pages to become extinct as they were replaced with new pages. When I resubmitted our site to the search engines, many engines said they would reindex the pages and drop the old ones. But as I watch the spiders come in, many of the spiders are still requesting the older pages (i.e. Ultraseek and they are getting the 404 file not found error. We have created our own 404 page to redirect the user to the home page to overcome this problem.
I thought I saw a post indicating how to tell the spiders that the page they are requesting should be removed from their db as it no linger exists. Can anyone help me find that post or let me know if there is a way in which I can do this?
12:42 am on Apr 18, 2001 (gmt 0)

Full Member

joined:Jan 24, 2001
votes: 0

In .htaccess file add the following line:

Redirect 301 /oldpage.html [yourwebsite.com...]

12:50 am on Apr 18, 2001 (gmt 0)

Senior Member

WebmasterWorld Senior Member mivox is a WebmasterWorld Top Contributor of All Time 10+ Year Member

joined:Dec 6, 2000
votes: 0

The 301 redirect will also direct your human visitors to the new pages, solving two problems at once.


2:58 am on Apr 18, 2001 (gmt 0)

Senior Member

WebmasterWorld Senior Member 10+ Year Member

joined:Jan 18, 2001
votes: 0

If you are giving the engine a true 404 page, with the proper status in the http header, that is the closest thing you can do to tell a spider that the page is gone.

To see the http header:

Start a DOS command prompt
From the command prompt type

telnet [microsoft.com...] 80

(Turn on localecho so you can see what you are doing.

Type Ctrl+] (hold down the Ctrl key and press the right bracket)


set localecho

Press Enter on a blank line

You only have to turn on localecho once.)


GET / HTTP/1.1
Host: www.microsoft.com

Press Enter twice