Static files served in development should not be cached

Django Internals
1

Currently, when Django serves static files in development it doesn’t include any cache headers, which allows the client to decide for itself whether or not, and for how long, to cache the resource. Instead, Django should serve these files with a Cache-Control: no-cache header to prevent caching.

This issue has been raised in several tickets (27572, 32891, 33148), and is marked as wontfix because, in brief, “We want to see the caching correctly set”.

The problem is that we don’t know how to correctly set the caching. Since Django doesn’t serve static files in production, that information exists somewhere else (an Apache configuration file, for example). Any given file could be designed to be cached forever, or never, or anything in between.

Given that, the only reasonable approach is to instruct the client not to cache files. Caching is an optional feature of HTTP, and not caching just means the server is receiving the requests it was designed to receive. Not caching can’t break your website, but caching files that were not meant to be cached definitely can.

Even if you think that static files should be cached by default in development, leaving off cache headers is the worst way to accomplish that. That allows the client to cache the files for an indeterminate and unreproducible amount of time, which is not what you want in a development environment.

(The tickets above are mostly focused on the issue of developer convenience. I think that’s another good reason not to cache - and presumably that’s why whitenoise disables caching in development even though it actually does know what the correct values should be - but the more important concern is the correctness issue.)

2 Likes
2

Hey @marfire :wave:

So, I actively do want the current behaviour. It’s me you’ve quoted there.

I’m not at all worried by the lengths of time argument, and all that jazz. Rather I want (and need) to see the behaviour with the browser fetching the assets from cache, which is the usual behaviour I need to see.

Often I want to turn that off. No problem, all the browsers have a quick access toggle to disable it.

Often I’m developing against an embedded web view in a mobile application. No problem, my browser makes the dev tools instantly available for all running web views, on all my development devices.

In neither of these latter cases does it merit setting up a web server in order to be able to observe the first, which is by far the most frequent case, and has no equivalent UI switch to turn it on, if were to be off by default.

This is why I’ve argued to maintain the current default, which is correct in my opinion. I hope that makes sense.

3

If you “need to see the behavior with the browser fetching assets from the cache” then you don’t want to leave off cache headers. That allows the client to do whatever it wants, including not caching anything at all.

The good news is that if you really do desire this behavior you can get it easily enough, no second server required. Just use the same monkeypatch that you’ve pointed out to others to change the cache header.

You didn’t address my point about correctness so let me expand on that. When the server uses no-cache the client is required by the standard to respect that. Sites can and do rely on that for their correct behavior. Imagine a site with an index.html and app.js, both of which are meant to be served with no-cache to ensure that they’re always in sync. Now the Django development server comes along and, in effect, strips the no-cache header, allowing the two resources to get out of sync. That could break the site, in arbitrarily bad ways.

The reverse isn’t true. A site that enables caching can’t be broken by the addition of no-cache, since caching is an optional part of HTTP and the server has to be ready to serve a fresh resource at any time.

When working production code is broken by the development server we should consider that a bug. Fortunately this is very easy to fix, and would bring Django’s behavior in line with whitenoise.

4

Sorry, I’m not seeing how you get to Django stripping the no-cache header? If you set a header it’ll be part of the response.

The current behaviour isn’t incorrect. It’s just not what you’d prefer as far as I can see. :thinking:

All your suggestion amounts to then is flipping the current default. Except I don’t think browsers have a UI button to enable caching even though there’s a specific header saying not to, so it would be strictly less flexible.

Unless there’s a new consideration I’m still against a change here.

The approach on the tickets seems pretty lightweight if you really don’t want the current behaviour.

I’m not sure what else I can say. :woman_shrugging:

5

I can’t find it instantly now but there was a ticket (or a discussion) about making static files serve better integrated into the middleware chain, rather than sitting beside it. I’m guessing maybe that’s the root of your complaint. It might be feasible to push on resolving that. (As a Saturday thought…)

6

But the header will never be set, that’s the point I think you’re missing. The correct header values are not available to Django, they’re off in an Apache configuration file, or an S3 setting, or some such. I’m not sure what ticket you’re talking about, but using the middleware stack won’t make any difference because the correct values are simply not in Django.

Imagine a static file in S3 configured to send a no-cache header. Clients are required to obey that header, and the site can depend on that for its correct behavior. In development, though, Django serves that file without a cache header, allowing the client to cache it, changing the behavior of the program. Sure, the development server didn’t take a response and literally strip off the header (that’s why I said “in effect”), but the outcome is the same. A program that’s completely correct and works in production can be broken when served in development because of the missing header.

If Django was able to serve static files with the production cache headers then this would indeed be a question of developer preference. But it can’t, in which case the only cache header guaranteed to preserve the behavior of the program is no-cache.

7

I think fundamentally we need to discuss who the users are of the software and what they want to do. I would argue the users are developers, and that they want to change their app, and after having changed their app, they want to use the app with the changes.

I am very active on the Unofficial Django Discord and every time anyone new asks anything at all about static files, you can pretty much blindly say “install whitenoise” and solve the problem. This isn’t a great situation…

2 Likes
8

I agree with the OP. It would be better if static files are not cached. Less flexible, but it is what you want in development 99 % of times. This could be configurable, but if the question is what the default should be, than no caching.

1 Like
9

I have a theory. Could it be that you are mostly developing applications that serve mostly static content (i.e. no JavaScript or none that often changes)? Then I kind of understand your viewpoint. Your application’s logic mostly lives in the server and you want to see a realistic representation of the behavior your app produces when interacting with a user’s web browser.

I, OTOH, am developing an application where a big part of the application is running inside the browser (~1k JS/TS files). So my dev environment is basically useless if it weren’t able to run the code I’m editing in the browser after a plain reload. Not using no-cache in my case would be similar to using ./manage.py runserver --noreload in your case.

I categorize not doing one of the following as a usability problem:

  • Disable caching while DEBUG is true.
  • Provide a setting to disable caching without installing additional dependencies or monkey-patching Django internals.
10

Is there a workaround to configure a specific site to never cache static files? If so I’d love to get that set up on demo sites I use for Django development.

I spend a fair bit of my livelihood doing UI development on Django projects and run into this all the time. A common problem is when doing cross-browser testing on mobile devices / browsers I’m not used to – those don’t necessarily have a clear “hard refresh” or “never cache” option.

11

@thibaudcolas - See the ticket #27572 (Static files served in development should prevent caching) – Django for a monkeypatch to django.views.static.serve

1 Like
12

Revisiting this – I’ve read the history of tickets and am having trouble seeing what is holding up this request. The original ticket #27572 was closed because of discussions about switching to WhiteNoise, which already implements what’s requested here.

Since then, over the four times this has been raised, I see 9 people requesting a different behavior than current – one “cache forever”, 8 wanting debug mode to “never cache” (counting myself here). And Carlton is the only person in those threads I’ve noted saying the current “no cache header set” is the right behavior.

So – how about a setting just like WHITENOISE_MAX_AGE?

  • Set 0 / no cache to prevent caching
  • Set to None to disable setting any Cache-Control header on non-versioned files.
  • Set to any other value if you do want caching of your files in local development

And it can default to “no cache” to match what seems to be the preference of a large majority of people?

Personally though I like having a monkeypatch available in a pinch, I can’t see myself ever wanting files to be cached in local development, and I’m also not going to start adding monkey patches to all projects I work on.

13

Thanks for raising this again, Thibaud.

I think your suggestion is certainly an improvement over the current behavior. Compared to just serving the files with no-cache, though, for which it appears there is rough consensus, I’m not sure a new setting adds a lot of value.

Note that WhiteNoise’s setting is one piece of a full production caching architecture. There’s also WHITENOISE_ADD_HEADERS_FUNCTION—which allows arbitrary changes to cache headers—as well as the settings of any CDN sitting in front of it.

By contrast, adding this in Django would mean re-implementing one small subset of the caching solution space to allow it to be tested using the development server. It won’t cover many caching approaches, and where it does the user will have to make sure that this setting stays in sync with their production cache settings for it to be a useful test. To me, a simpler and more coherent approach is to limit the development server to testing the correctness of your Django code and leave the testing of your caching architecture to something else.

So I guess I’m +1 on simply adding no-cache to static files and +0 on this alternative.

(On a technical point, you mention that some behavior will apply to “non-versioned files”. Does that mean that we also need an equivalent setting to WHITENOISE_IMMUTABLE_FILE_TEST so that Django can determine what is a versioned file or not? Or were you imagining this to just apply to the ManifestStaticFilesStorage versioning algorithm?)