Implement and test cache busting

Using Django Deployment
1

It looks like I’m facing cache issues so I would like to implement this cache busting feature as it was recommended to me. As I’m not comfortable at all with all of this stuff and static files management, I would like to double check the process and understand how I could test it before pushing to production.
The context: my application is already in a production server (alpha version) and as this is an evolution of the ground settings, I would like to take this opportunity to train myself to prepare, test and deliver such technical updates. I’ll take the opportunity to better understand static files management. Currently, the js and css files are not properly propagated and my production app still uses older versions, so I need to enforce a server cache refresh.

How to proceed
As far as I understand, what I have to do is to change my settings file and add the following line:

STATICFILES_STORAGE = 'django.contrib.staticfiles.storage.ManifestStaticFilesStorage'

I’m wondering if I need to change anything else. Here are my current settings about static files management:

STATIC_URL = "/static/"
STATIC_ROOT = os.path.join(BASE_DIR, "static/")
STATICFILES_DIRS = []

I’ve mainly seen several approaches for STATICFILES_DIRS (either [] or ./myapp/static or (os.path.join(BASE_DIR, "my_project", "static"), )) but I do not know the differences, and what is the better setting

I’ve read some use cases where it was recommended to ‘reinit’ the settings: delete the static directory and rebuild everything (migrations, collect static), and I wonder if it’s necessary? Shall I run python manage.py collectstatic? Anything else?

How to test
The key parameter to activate this feature is to set DEBUG = False
Can I test this modification directly in my local development environment? To check if it works, is it enough to make small changes within js / css and ensure these changes are applied?

How to deploy
Are there any dedicated manipulations to perform?
I mean, I will pull the committed changes and reload gunicorn service as usual), am I supposed to do anything else?

Many thanks for your advises, again I’m not comfortable with such subject and I hope it will help other newbies like I am ^^

2

This setting controls where your static files come from. If you use STATICFILES_DIRS = [], you’ll only be using static files from within your apps, if you’re comfortable with that.

It’s not necessary to delete files in your STATIC_ROOT, but they will accumulate there over time on your production server - if you aren’t using new servers/VM’s/containers per deploy. You can use the --clear flag.

STATIC_ROOT should not be committed to your source control. Instead you should populate it on every deploy by running collectstatic, at the same time you run migrate.

Yes you can test it. Turn off DEBUG and run collecstatic and runserver will show you what that looks like.

Again, don’t commit the collected static files. So that sounds like the biggest change to your process.

Consider using WhiteNoise. It goes beyond django.contrib.staticfiles to also serve the files from within Django, so you don’t need to configure a web server like nginx to do so. It also sets correct headers and provides compression, which are things that you might easily miss when configuring a web server.

3

Many thanks Adam!
If I understand good, STATICFILES_DIRS lists all static files sources, and collectstatic will gather them at STATIC_ROOT directory and make them available (a kind of referencing) by Django app?
I will also consider implementing WhiteNoise, but for this app I do not have so many files to manage (event if does not look that complicated to set up)

4

Not quite accurate.

collectstatic will copy all your static files to STATIC_ROOT. But the (original) intent was to get them out from your Django app and make them available to be served directly by your web server (e.g. Apache httpd or nginx, etc).

The idea is that Django / Python really isn’t needed to serve static files. Software designed to cache and send static files is more efficient to perform that function than an application. This line of thought can be extended to include the distribution of those files to something like AWS S3 or a CDN.

See the docs at How to deploy static files | Django documentation | Django for more details.

5

Well, to be honest I do not really understand :-/
In the meanwhile I made the update locally,n then ran collectstatic: I do not see any changes, and I’m not able to run my application locally (I tried different options for ALLOWED_HOSTS but I have either Error 400 or Error 500.
I’m quite sure the original problem is different, but after running collectstatic, I got many duplicated image files in my media directory (I just cleaned this folder before).
My *.js and *.css files have correctly been renamed with any MD5 hash… but I now have too many new files, according to the above comments, what am I supposed to commit?
I’m a bit lost :-/

6

Ok, I’m starting to get the impression that we may be conflating a couple of different issues here.

In no particular order:

  • Collectstatic really isn’t used or needed in your local development environment, unless you’re testing with what I refer to as a “production quality deployment”. (Typically, your development environments are going to be running with DEBUG=True - and for a good reason.)

  • 500 Errors should be addressed separately and independently of any other issues you may be working with.

  • Your media directories should be separate and different from your static directories. They are managed and controlled differently.

  • You don’t commit the hash-reference files. Collectstatic is something you typically run in your deployment environment, not in your development environment. (See point #1)

7

Many thanks for these explanations!

It’s also my fault, I’m quite new to these subject and these very technical topics are not my favorites ^^

Well, my bad, I did it to test the main subject - cache busting implementation.
I think I will just delete the files that have been created, I will run collectstatic in my production environment (I’ll forget these 400 / 500 error and come back to DEBUG=True in dev environment for the time being).

I’ll probably need to open a dedicated topic to discuss this subject - I will review the documentation before

8

A quick feedback here to thank you all: I made the changes on my production server and it looks working fine.

To be honest, I was frightened at the beginning because my application did work anymore at all! By chance, I implemented Sentry that gave me the reason: I had an old js file I put in comment in my base.html, and of course was deleted from the server. But with this set up (or because I ran collectstatic), it just blocked everything.
It’s all in order now.

Thanks again

9

I am also getting same error(500). May I know how you solved it?