nginx proxy_cache and If-None-Match

1. You mention doing http://nginx.org/r/uwsgi_pass, but then doing caching with http://nginx.org/r/proxy_cache et al, not with http://nginx.org/r/uwsgi_cache et al. Are you sure it's actually working properly? I wasn't aware that these directives are interchangeable — I don't think they are at all, even though they certainly sound similar and do similar things.

2. Have you tried changing http://nginx.org/r/if_modified_since from the default of exact to off?

   if_modified_since off;
   

   Based on my reading of src/http/modules/ngx_http_not_modified_filter_module.c :: ngx_http_not_modified_header_filter():

   • If the header If-Modified-Since is present (and is non-empty), and the if_modified_since directive is set to off (being NGX_HTTP_IMS_OFF define in the source code), then ngx_http_test_if_modified() immediately returns true, resulting in the filter shortcircuiting with return ngx_http_next_header_filter(r); in other words, in such a case, the If-None-Match logic wouldn't be executing.

   • Note that this happens only if the If-Modified-Since header is actually present (and non-empty) in the request (in addition to If-None-Match that you're dealing with), otherwise, the code path will continue without the NGX_HTTP_IMS_OFF setting having any effect; I think the expectation here is that if the clients already include If-None-Match, then If-Modified-Since is likely included as well; thus, there doesn't seem to be a separate option to disable processing of If-None-Match specifically and individually.

     78    if (r->headers_in.if_modified_since || r->headers_in.if_none_match) {
     79
     80        if (r->headers_in.if_modified_since
     81            && ngx_http_test_if_modified(r))
     82        {
     83            return ngx_http_next_header_filter(r);
     84        }
     85
     86        if (r->headers_in.if_none_match
     87            && !ngx_http_test_if_match(r, r->headers_in.if_none_match))
     88        {
     89            return ngx_http_next_header_filter(r);
     90        }
     
     …
     
     132ngx_http_test_if_modified(ngx_http_request_t *r)
     133{
     …
     139    if (clcf->if_modified_since == NGX_HTTP_IMS_OFF) {
     140        return 1;
     141    }
     

   • Note that ETag support, together with the If-None-Match, is only available in stable-1.4 and above (it's not present in stable-1.2 and below), so, downgrading might also be an option.

3. Have you tried clearing the ETag before the client?

   • Do your clients send If-None-Match: *? If so, then as per https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.26, not performing the GET action would be correct (e.g., returning 304 Not Modified is entirely correct), and that's exactly what nginx does, which can easily be verified by running something like curl -H"If-None-Match: *" -v localhost:80 on a default installation with nginx/1.15.9 (note that stable-1.2 doesn't have ETag or If-None-Match support, as they first appeared with stable-1.4).

   • If not, you might try to omit the ETag in your responses to trick the clients to not send an If-None-Match.

     You can use http://nginx.org/r/add_header to clear the ETag header from responses to the clients:

     add_header ETag "";
     

   BTW, I've also tried modifying $http_if_none_match directly with http://nginx.org/r/set, but it didn't have any effect.

4. As a final resort, if a better solution isn't available (although I think that the above solutions might as well work), what you could do is put another instance of nginx in front of your regular caching nginx specifically for those buggy clients. In this new nginx instance, you'd disable all sorts of caching, remove the header that's not handled correctly by your buggy client apps, and pass the request to the normal instance of nginx that does do the caching.

   More specifically, you might want to make sure that this new instance wouldn't do any buffering, either, http://nginx.org/r/proxy_buffering, otherwise, you risk getting your performance reduced by an extra step of saving stuff to disk all over again by the extra instance; else, it should all be copied in memory, and will likely be fast enough.

   I think this approach should definitely work, because this frontline nginx wouldn't have any cache, thus, it couldn't possibly reply with 304 Not Modified, whereas the backend nginx wouldn't receive the If-None-Match if you clear the If-None-Match on the frontend nginx with http://nginx.org/r/proxy_set_header before doing a http://nginx.org/r/proxy_pass to the backend nginx.

P.S. The other answer suggests using http://nginx.org/r/proxy_ignore_headers, but, based on documentation around it as well as around http://nginx.org/r/proxy_set_header, it doesn't seem like sprinkling If-None-Match should make any difference; plus, it looks like there's no _set_header variant for uwsgi anyways.

P.P.S. Another potential avenue, if you don't actually need to deliver things from cache, is changing http://nginx.org/r/uwsgi_cache_revalidate from the default of off to on; however, I think this, too, would depend on _set_header, which is missing in the uwsgi version.