I manage a site with a lot of temporary, limited time user generated pages that expire. Right now we keep the pages, telling the visitor that the information has expired.
But I consider handling this a new way because:
• We have a big load of new user generated content every day. Theoretically it becomes a huge number of pages on my site over time. (theoretically because in practice it does not all get crawled and indexed by Google).
• It will take a lot of resources for Google to crawl all these "old" pages, and thus it takes away resources for the crawl from other more important things on the site.
• Google can/will not keep all these pages in its index and simultaneously index all of our new content. We face that problem now.
• I guess the easiest way to solve this would be to remove the expired pages and return a 404 or 410 header response to Google. So they would be removed from the index and not crawled again.
• But this way you will lose all the link juice that is created when people link to their own content from other sites.
• So my best guess is to do a 301 redirect to a search result page from a similar search. This way you keep the link juice within your site. When the user (coming from the search engine) comes to this page, you could eg. give a small pop up that says: "This content has expired but here is a lot of other pages with similar ......". This pop up comes naturally only when the user was redirected from an expired page.
But I am a little worried about the large number of redirects will have negative effect in Google and I don't know if there are any other disadvantages. How should I handle this? What is your suggestion? Thanks in advance.
[edited by: tedster at 11:24 am (utc) on Aug. 28, 2009]