Planet13 - 1:33 am on Aug 17, 2010 (gmt 0)
The result of all this is the spider wastes b/w from me on the one hand and on the other, wastes its resources instead, it could crawl a valid page.
I gotta' ask...
Is there a way to make it into a "valid" page?
By that, I mean making it into a page that will somehow help you generate revenue?
I have some old (discontinued) products on my ecommerce site and used to have them redirect to the home page (which has been pointed out is NOT something you should do). So I am going to redirect them to a "product archive" page (which would be the same as the original page except that it clearly states the item is no longer available) and then add links to the most relevant products that ARE still available.
Agreed, it would be better if google spent its resources crawling only the pages we want it to, but on the other hand, if the page still has the potential to bring in visitors, I think there might be a way to leverage that.