I am looking at this settings.py snippet:
# settings.py
if APP_NAME:
FOO_COLLECTION = f'{APP_NAME}-jobs'
else:
FOO_COLLECTION = env_var('FOO_COLLECTION', 'jobs')
I don’t like it.
There are three cases. That’s too much.
I have a rough idea: Make this in two steps:
Step1: Create some config in JSON format.
Step2: Apply this config conditionless (without “if” and “else”).
I am not looking for what people call configuration-management like Ansible or Chef/Puppet.
I am looking for a simple way to have a tree of sane defaults, with well defined input and output.
Above snippet has three cases. It is untested code. It gets tested by hoping it will work in a new environment. I don’t trust myself. I want to have tests for this.
If I separate this into two steps, then I could test Step1 easily. And the second step is just
a simple assignment which needs no test.
Is there a name for my proposed solution?
I am missing the right term to google to see how other people solve this.
You could try a defaultdict and use the condition as key and the json as values.
I’d like to challenge your assumption that you cannot test conditionals in your settings files. It’s very much possible, with a little help.
The problem is that the code runs at import time. It would seem we can import the settings only once during a test run, but that’s not true. We can use Python’s importlib to reimport a module, using a recipe it spells out. To make it easier, we can write a wrapper function that also ensures we reimport with a unique name, based on a counter:
from importlib import import_module
from importlib.util import module_from_spec, spec_from_file_location
from types import ModuleType
reimport_module_counter = 1
def reimport_module(name: str) -> ModuleType:
global reimport_module_counter
counter = reimport_module_counter
reimport_module_counter += 1
original = import_module(name)
# Recipe for importing from path as documented in importlib
spec = spec_from_file_location(
f"_remiport_module.{name}_{counter}",
original.__file__,
)
assert spec is not None
module = module_from_spec(spec)
# typeshed says exec_module does not always exist:
spec.loader.exec_module(module) # type: ignore[union-attr]
return module
We can then test our settings by reimporting them in various conditions, e.g. with particular environment variables set:
import os
from unittest import mock
from django.test import SimpleTestCase
from example import settings
class SettingsTests(SimpleTestCase):
def reimport_settings(self):
return reimport_module("db_buddy.settings")
def test_debug(self):
with mock.patch.dict(os.environ, {"DEBUG": "1"}):
module = self.reimport_settings()
assert module.DEBUG
assert "debug_toolbar" in module.INSTALLED_APPS
assert module.INTERNAL_IPS == ["127.0.0.1"]
Hope that helps
2 Likes
Thank you Adam for this alternative solution.
Nevertheless I would somehow like it, if there were would be two steps:
Step1: Generate the config
Step2: Apply the config in a conditionless way. So no “if” and “else” in settings.py
I’m not following you here at all.
You have a setup where a system setting is determined at runtime. You’ve got three different possibilities where you don’t know what the results are going to be before you start the server.
All I’m seeing from what you propose is an additional step to make that determination prior to starting uwsgi / gunicorn / whatever. I don’t see where you’re eliminating that condition, you’re just moving it elsewhere. In my mind, that’s just adding to the complexity, not simplifying it.
If you really aren’t deploying multiple instances where something needs to be checked when the project is starting, then you can remove the conditions and replace that in your settings with this:
FOO_COLLECTION = env_var('FOO_COLLECTION')
and require that FOO_COLLECTION be set in the environment appropriately.
@adamchainz
we like your approach that you also include in your great book “Boost your Django DX”. Hint: of course we acquired both of your books 
We follow the settings convention described in Django Tips #20 Working With Multiple Settings Modules and have multiple settings files.
Unfortunately we run into the following issues:
======================================================================
ERROR: test_check_s3_turned_off (zemtu.tests.tests_settings.SettingsTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/opt/zemtu/app/zemtu/tests/tests_settings.py", line 45, in test_check_s3_turned_off
module = reimport_module("zemtu.settings.development")
File "/opt/zemtu/app/zemtu/tests/tests_settings.py", line 32, in reimport_module
spec.loader.exec_module(module)
File "<frozen importlib._bootstrap_external>", line 883, in exec_module
File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
File "/opt/zemtu/app/zemtu/settings/development.py", line 11, in <module>
from .base import * # NOQA
ModuleNotFoundError: No module named '_remiport_module'
----------------------------------------------------------------------
What solution do you recommend?
Best,
Dominik
Your response is quite off-topic for this old thread, you should have made a new one!
Thanks for buying my books.
Your failure is happening because you’re using a relative import, which is being resolved against the temporary _remiport_module.{name} (btw fixing that typo). You could try switching to an absolute import, but that will be imperfect because it will use a cloned development module but the single base module.
The testing strategy is designed for a single settings file approach because it reimports a single module. Ideally, you would adapt it to reimport your base AND target (development) module… or move to a single module approach. If you do try testing your multiple file approach, beware this issue: A Problem with Duplicated Mutable Constants - Adam Johnson .
Good luck.