A proposal for the fresh bot

add "nofresh" to the googlebot meta tag

[Sasquatch]

I would like to see "nofresh" added as an option to the googlebot meta tag. <meta name="googlebot" content="nofresh">

I already sent this suggestion to google, but I was hoping that some feedback here might cause googleguy to notice and push for the addition.

The majority of my content pages will not change more than once every couple of months. But the navigation portions of the page (php generated) might change often enough to trigger the freshbot. Having these pages crawled by the freshbot does not do anyone any good, I would much rather have google spending the time to search sites that would have real minty fresh content.

I would also prefer that my logs represent real user hits instead of having googlebot taking an interest in some obsecure page that would only get a few hits a month on it's own and skewing the results.

Of course there would be "nofresh"'s compliment, "fresh". But we all know that it would be pointless for google to actually pay attention to that as soon as webmasters started abusing it, which would be within days of it being implemented.

[Slade]

I recently added this to my php on my personal site:
[php]
header("Last-Modified: " . date("r", $page["lastmodified"]));
[/php]
Of course, I've already populated $page["lastmodified"] with a unix timestamp.

Somebody help me if I'm mistaken, but wouldn't this cut down on refreshes, if they can tell that content doesn't change often?

[GoogleGuy]

I'll pass this suggestion on, Sasquatch.

[Sasquatch]

Thanks GoogleGuy.

Slade, the lastmodified might keep it from actually retrieveing the page, but it will still get you a hit. It can also cause a user to not have my most recent navigation info if their browser were to pull up their cached copy. It is a good idea though, and would work in a lot of cases.

[Slade]

I can see that, and have been thinking about updates to my menu structure for this site. I don't know what exactly to do there, except maybe remove the timestamps one month or something...

ps: I think there has some kind of alert system on here that attracts GoogleGuy... maybe his pager goes off everytime someone posts his name...

[martin]

Slade the RFCs say that you shouldn't actually use date('r');

Use date('D, d M Y H:i:s', $date) . ' GMT' instead.

[Slade]

Can you give me a hint what you're talking about?

From the php date page:
[php]
r - RFC 822 formatted date; e.g. "Thu, 21 Dec 2000 16:01:07 +0200" (added in PHP 4.0.4)
[/php]
From www.apache.org:

Date: Tue, 08 Oct 2002 18:13:37 GMT

From www.google.com:

Date: Tue, 08 Oct 2002 18:12:15 GMT

From my site:

Date: Tue, 08 Oct 2002 18:13:08 GMT

[martin]

For me gmdate('r'); is

Tue, 8 Oct 2002 18:21:44 +0000

so it probably depends on the locale settings. I don't thing risking with such a thing is a good idea.

[Slade]

I can see your point, and it appears I just got lucky...

This page: [php.net...] shows a little bit of discusstion on this, and gives some examples.

I have to make a few other changes to the script I'm building, so I'll include it at a later date. Fortunately, I am doing it in a script, so I only have to update it in one place :)