Mediabot and HTTP headers

How to keep all involved happy?

[ExpLarry]

It's a fine sunny day here in Webmasterland so I'm sitting on the veranda (yay for WLAN) watching logfiles scroll by. (Note to self: get a life).

As things are a bit slow I can see what pages are bringing the Mediabot in, and it seems it is happy to revisit a page shortly after that page was accessed, even though the page is displaying relevant ads, and the page's HTTP headers indicated the page itself was updated recently and will not expire for few days.

IMHO frequent Mediabot visits are a Good Thing (TM), keeps the ads up-to-date and all that, but I was wandering whether there's any optimal configuration of HTTP headers and the like which give the Mediabot hints about how often it should revisit, which saves me and Google bandwith, CPU cycles etc.

Ideas?

[linear]

Seems to me we had a thread on "does mediapartners-googlebot respect/use the if-modified-since header?" and the answer was that we really didn't know. Maybe I'm recalling incorrectly though.

I'd think that this would be an excellent opportuninty for ASA to commment (and on a non-emotional issue too! :D )

[Jenstar]

There doesn't seem to be any rhyme or reason to when the mediabot pays a visit, unless you trigger one by changing the ad unit size. AdSense is mainly for pages that the content would not change on a regular basis.

AFAIK, the regular googlebot ignores those headers as well, and also arrives on its own schedule ;)