Caching instructions in HTTP headers

Cache-Control, Pragma

[stanislawek]

I am exploring cache-control mechanisms for a client and believe that Google is not following the HTTP specification.

I have added cache-control, pragma and expires http header directives (note: not meta tags) to http responses to prevent caching of content. I am suprised to see that Google still provides a cache link for pages returned with these response headers on their search results page. The objective is to have Google index the page but not provide a cache link.

I am aware this can be accomplished via Google's meta tag for no archiving, but based on my understanding of http, the directives embedded in the http response should disallow caching by proxies (or Google).

Any insight, comments or suggestions is appreciated.

[Brett_Tabke]

Although I salute the thought, the two are not related in any way. Other than content-type, most don't believe G is using any http headers at all.

There was some talk a few months back about the possiblity of spidering frequency being related to http expires headers. No one can say for certain if G is using them or not (I don't think so).

The biggest problem is the error rate in http headers. Try running a spider some time. I'd estimate that 15-30% of the sites out there have something wrong with their headers.

[daroz]

The biggest problem is the error rate in http headers. Try running a spider some time. I'd estimate that 15-30% of the sites out there have something wrong with their headers.

And every one of my sites will have this problem. Thank you IE 5.

Another possible issue is many sites want to expire content immediately (to prevent the back button from displaying stale content). If Google were to all of a sudden start relying on that data many many sites I fear would fall out of the Google Cache, even though they want in.

The Meta tag isn't a big deal to put in, and it prevents alot of these issues.

[jdMorgan]

stanislawek,

You are correct. Google's cache is not affected by HTTP headers, because they do not consider it to be part of the network. They consider it to be an archive, and it is, since it has a unique and separate URL from the actual resource.

The HTTP cache-control headers will affect only caches between the server of the requested page and the user's browser. The noarchive meta tag is the only way to stay out of the Google "cache" if you want the page to be indexed.

HTH,
Jim