Site Caching

always out of date

[Dave McClure]

Greetings.
I am living in the Middle East, where there is considerable censorship of the web. My own material is not 'adult'. I use blogger to publish the index.html homepage to my own domain. The rest of the site is 'conventional'.

If I navigate to http://www.example.co.uk, I always see a cached page, maybe 5 or 6 days old. It is not cached on my pc, as it comes back after I clear the cache and try again. Also, that's the page I'll see from any pc in the area.

But if I navigate to http://www.example.co.uk/index.html, I will get the new page every time, even minutes after posting it. This is fine for me, but presumably other readers will always be seeing the old cached version.

Is there any way I can remotely flush or update a cache that I don't own? Presumable it is actually hosted by the monopoly telecoms company of this Emirate.

[edited by: encyclo at 5:32 pm (utc) on Oct. 6, 2007]
[edit reason] switched to example.com [/edit]

[jtara]

First, try the obvious - there are headers and metas for cache control. Try them. Particularly, the "Expires" HTTP header, and the newer "Cache-Control" header.

As an aside, from your description, this is not a traditional proxy server - that is, the browser is presumably not set to use a proxy server - it's set to access the web "directly". A router or other device that can "filter" TCP/IP streams is intercepting traffic and satisfying HTTP requests from a cache.

You seem to have uncovered a clue for a workaround, though. You just might not realize it. :) You've discovered an area of non-coverage.

Try a 301 or 302 redirect to index.html. (Experiment with different kinds of redirects. Heck, maybe even redirect to an "error" page - that isn't one.)

Alternately, why not just give instructions (not sure it would be a good idea to actually include a clickable link) for users to type in "example.com/index.html"?

Of course, this may not last long, but might get you a few weeks/months/years of visability.

[Dave McClure]

jtara -
Thanks for that detailed response. Lots to try there. I'll do some experimenting.

There's certainly something unusual in the path. If I do a tracert, before it leaves Qatar there are three steps:

if-2-9.csb-aggr.qatar.net.qa [IP addr]
if-3-2-1.csb-core1.qatar.net.qa [IP addr]
if-3-0-0.csb-gv2.qatar.net.qa [IP addr]

Then it sets off to the UK via UAE

[jdMorgan]

Following up on jtara's post, I'd like to clarify that you, as the Webmaster, control caching behaviour by configuring your server to send the Cache-control and Expires HTTP response headers when serving your content. If you do not specify these headers, then is is up to the client browser and any intermediary caching proxies to decide how long to cache your content. The typical default is one-half of the time between the current request and the last time the page was modified -- This based on the Last-Modified server response header.

You can use the "Live HTTP Headers" extension for Firefox and Mozilla browsers to view real-time HTTP request and response headers. Examining the server response headers for the two different cases you cite may reveal an important difference in the responses for the requests for the two URLs.

Then, based on the server software your site is hosted on -- e.g. Apache or IIS, you can search for the information you need to configure your Cache-control and Expires headers.

Jim

[Dave McClure]

Thanks Jim -
I've got a lot to learn here. I always validate my html & css and put in a few descriptive meta tags, but I've never looked far into the server side stuff. I guess it's time to start!

[jtara]

I'd imagine that most webmasters never think about or fiddle with their cache control headers, and get along just fine.

When your target audience is in a country or countries where caching (and/or filtering - because if you're filtering why not cache while you are at it, and lower the infrastructure demand?) is common, it suddenly becomes a priority.

[Dave McClure]

Both -

As often happens here, your answers sent me off on a research trip, during which I came across an article that seemed very good for someone at my level, though maybe 'old hat' to the experts. For the record,here it is:

[mnot.net...]