mod cache: how to cache http 500 reply too?

[Andy16]

Hi out there

I am running an apache instance with mod_cache and cache_disk_module if front of my backend to deduplicate requests for 30 sec.
So every new request reaches the backend, gets cached and will be answered from cache for 30 secs.
Basically that works fine.

Sadly some requests are bad, hence httpd 500 from backend - but errors alike do not get cached!
How can I have the same behavior on 500 like on 200?

[httpd.apache.org...]
I tried hints in tons of combinations - none gave expected / usable result...

any help/hint appreciated

[graeme_p]

500 errors should be fixed, not cached!

[Andy16]

:-)
in general you are right - but also requesting the same page and failing again and again is of no use but additional effort

thanks for help

[graeme_p]

I do not think mod_cache does this.

You could put a proxy that can be configured to cache 500 errors in front on Apache. I think Varnish can, for example.

[Andy16]

thanks a lot
Varnish would bring me a new stack, currently trying still to solve on apache level.
if mod-cache cannot do, any other idea to address it by combinations of other moduels e.g.?

[phranque]

welcome to WebmasterWorld [webmasterworld.com], Andy16!

is it a known/finite list of urls or are they arbitrary?

[Andy16]

ok, need to give more details...

Backend is da "data harvesting engine" to query data from external devices.
Which devices to scrape -> part of the url from requestor.

Now requestor can go nuts and request data really often - fine - will hit the cache and not bother the engine neither the target device.
It may also happen that target device is down, not reachable, what so ever, and backend will reply 500 => requestor will try again, again, again
.... and backend will have to retry, retry, retry
==> this is the situation where I want the cache to kick in and do same as for valid links.
So links are arbitrary and error situation is limited in duration.

So in the end in "only" need to handle error messages like normal replies and cache them too.

I considered that a valid and easy request, until I read RFC, man pages, how to's, .... :-(

[phranque]

i believe that you need to solve this by having the backend provide cache control headers with the 500 response:
https://www.rfc-editor.org/rfc/rfc7234#section-2

[Andy16]

this is the RFC which made me understand my request is not really "RFC compliant"... :-)
Problem: backend is a closed system I basically cannot change.

The only idea I at present can imagine is to have another proxy inbetween that changes error messages like 500 to 200 (which cache will accept).
Can I tweak apache mod_rewrite (or similar) to do so?

[phranque]

The only idea I at present can imagine is to have another proxy inbetween that changes error messages like 500 to 200 (which cache will accept).

i think you can do this all with mod_headers:
https://httpd.apache.org/docs/2.4/mod/mod_headers.html#header

perhaps something like this is sufficient:

Header always set Cache-Control s-maxage=30 "expr=%{REQUEST_STATUS} == 500"

[graeme_p]

A 500 response looks wrong to me in this case. Send a 503 as it lets you send a retry after header or possibly a 202 or even a 204. You can then set a cache control header as phranque suggests just for that code (so proper 500 responses are not affected).

[Andy16]

@phranque - testing on the Header always set Cache-Control, thanks for the hint
@grame_p - as mentioned above, backend I cannot adjust or change, not in my hands