Validating settings file on build instead of deploy

Using Django Deployment
1

I am new to Django, so I apologize in advance if this is a dumb question for whatever reason. We recently had an incident where there was a misconfiguration in our settings.py file for one of our prod services. Basically we had some code in there to pull a secret from GCP secret manager and load it into an environment variable, however the key in the actual secret was mistyped so it threw an error saying no environment variable with that name could be found.

The issue is this happened when the service was actually being deployed (to Cloud Run), which broke the deployment and left our service in a weird state. I am looking for a way to validate all of the environment variables for a specific settings file earlier in the process so it fails in the CI/CD process of our Github PR flow, so people can’t merge in changes that will break the deployment since their PR won’t succeed the build steps.

Our actual settings files are broken up into dev, prod, and test, and the test settings file is what is used in our testing step of our CI/CD pipeline, and that settings file had no issues, so we are looking for a way to also validate the prod version of the settings file.

2

This is likely going to be something that you would need to build yourself. I can think of many situations or settings that simply can’t be verified outside their target deployment environment.

If you can maintain a copy of what those critical settings need to be in a “local” storage area, you could at least verify that the settings being deployed match what you need them to be.

But any setting effective only in your production environment can (almost by definition) only be verified there.

What you might be able to do is to create a simple management command that attempts to verify what those settings and include that in your project. Then run that command as the first step of your production deployment process to verify that the settings are “good” before bringing up the rest of the system. (Obviously, the specifics of this are greatly going to depend upon how you do your deployments.)