Welcome to WebmasterWorld Guest from

Forum Moderators: open

Message Too Old, No Replies

Time Sensitive Content and Googlebot

Time Sensitive Content and Googlebot



12:21 am on Jan 6, 2003 (gmt 0)

10+ Year Member


I have a site that shows dynamically generated content that becomes stale at a specified date in the future.

What I've been doing so far is when the page expires to redirect Googlebot (http - user agent) to the home page and to human users (coming as Google referals) show the original page with a warning that the content may be out of date. (From a bussiness perspective it's better to show humans the content rather than a 404)

I realize that this is not an optimal solution.

Would it be better to give crawlers 404 and users the page with warning, or when the content expires add <meta robots - noindex>?

Or is there any better solution?




12:34 am on Jan 6, 2003 (gmt 0)

Is there a problem updating your dynamic content before it expires? I would say that is the easiest way to correct the problem.

I would defiantly stay away from showing googlebot one thing and your users another - this is a sure way to arouse the wrath of Google and get yourself a PR0


12:52 am on Jan 6, 2003 (gmt 0)

10+ Year Member

I have new content coming in and expiring almost every day. Think of it as this message board except with time sensitive content.

The whole purpose of what I'm trying to do is to get Google to drop the stale pages and pickup new ones.

Thanks for your help.


12:52 am on Jan 6, 2003 (gmt 0)

WebmasterWorld Senior Member macguru is a WebmasterWorld Top Contributor of All Time 10+ Year Member

Hi Vlas_L,

Welcome to the board. I see content as some expensive good. Hard to throw away for both bots and visitors.

Here is the simple thing I do for expired content on some sites. I warn the visitor using text on a bigger than banner sized gif linking to fresher version. It only adds a line of code to expired pages, no one gets 404 and fresher content is easily found.

This requires unique URL for each documents and more efforts updating links to current versions. But it pays where I use it.

I heard Googlebot was following meta refresh, not sure...

Hope this helps.


Featured Threads

Hot Threads This Week

Hot Threads This Month