What to do with old files spidered by google bot?

[silverbytes]

Googlebot is spidering old files that are not anymore in server... I see in log accessing

/directory/page.htm

which is not on server... how is that possible?
In fact accessed index.htm, robots.txt and that file only...

Any idea?

What is best for deleted documents? to redirect permanently to a general document in the site? Just delete it? Since it will be in google's search results but not on the server...

[edited by: WebGuerrilla at 8:36 pm (utc) on Sep. 24, 2003]
[edit reason] widgetized [/edit]

[WebGuerrilla]

As long as your server is sending a proper 404, your best plan would be to simply delete any old files you don't want and then forget about it. Google will continue to request the old urls for awhile, but they will eventually stop.

[silverbytes]

If I put a permanent redirect to index wouldn't be better?
I mean, users go away on 404 pages (even in custom many of them run away).
...

Or that causes the page to be in google's result and never dissapear?

[closed]

I'd use either a permanent redirect (301) or gone (410) in your case. See:

[w3.org ]

[berli]

Can anybody quickly give an example of how to do a 410 "Gone" on an Apache server (.htaccess)?

[closed]

You would use RewriteRule just like you would with a forbidden page, except that instead of using the F flag for a forbidden page, you'd use the G flag for a page that's gone.

For an example of the use of the F flag, see this page:

[httpd.apache.org ]