What is this warning for AppConfig.ready() about?

Using Django
1

In the documentation of the AppConfig class there is a warning that reads:

Although you can access model classes as described above, avoid interacting with the database in your ready() implementation. This includes model methods that execute queries ( save() , delete() , manager methods etc.), and also raw SQL queries via django.db.connection . Your ready() method will run during startup of every management command. For example, even though the test database configuration is separate from the production settings, manage.py test would still execute some queries against your production database!

The problem is that this is precisely what I need to do at this point, so I’m trying to understand what could go Horribly Wrong ™ if I ignore the warning. We need a system to dynamically load/modify database connections and our current solution (a cache of connection details in a file that is read by the settings) does not sit well as it involves cleartext passwords in a file.

So what I did was set a breakpoint in a dummy AppConfig.ready() implementation and check the state during the call to the method. At this point settings.py has been imported and both it and the django.db.connection shortcut point to the database I was expecting, namely the test database when the server was started via python manage.py test.

In our case the settings automatically adjust the default database connection to point to a production* or test database depending on which command started the server. But even if you split the settings into seperate files and control which to use via how the server is started, in both cases the default database connection points the right direction by the time AppConfig.ready() is called.

What am I missing here? Where’s the danger?

* I don’t know why you would test in a production environment, but let’s assume that makes sense here.

2

For the situation where we need more dynamic control over the database connections, we have that initialization code in the settings file.
(We dynamically modify the DATABASES dict by calling a function that makes an external API call to a resource. There are other settings that get changed / managed this way as well.)
Your settings file is Python code, it can be abused in many interesting ways…

3

I agree there are interesting and dangerous things to be done in settings.py. My reasoning for piggybacking on AppConfig.ready() is that if it is dangerous to read from the database there, this should go doubly so for the settings. I just don’t understand why it is dangerous.

Your solution of using an external resource is also interesting, but seems a bit overkill in this case, especially as we will need the option to add new connections to a running server anyway. I got a solution in about 50 lines of code, but it leaves me wondering if I haven’t created the metaphorical pit trap without realising it.

Anyway, your input is always appreciated, thanks.

4

Opinion: I think it’s less so, because there has been no database initialization at that point. You would have to completely manage the connections independent of any Models or system connections. It’s something completely outside Django’s management.

Neither do I. It would be interesting to see more about this.

Probably is for you - we had a very odd situation we were dealing with, and it was the best solution we could come up with at the time. I only mentioned it in that the same idea may work if that script made a database connection or read a file rather than making an API call.
The idea was to find a place as early in the initialization process as practical that we could hook in to, so that the rest of the initialization process would be “clean”. (Our alternative was to write a wrapper completely around the entire process to set environment variables before starting uwsgi, but that ended up being more awkward than it was worth.)

5

FYI there’s a ticket to warn, and eventually ban, queries until after all AppConfig.ready()s have been called: #33143 (Block import-time queries) – Django . Most of the time, such import-time queries are a mistake.

6

In light of the warnings against queries using the ORM before the apps are ready, we decided to go the route of doing it all in the settings as well. While it does result in settings code that is more brittle than desired, it has the upside that any broken code is made painfully obvious to the developer as the server won’t even get to the point of throwing hundreds of failed tests due to missing database connections.

The process is now this:

  1. Set default values for non-essential settings
  2. Load custom values for settings, which in turn:
    1. Check if the executed command is the test command.
    2. Set the default database to either a SQLite test database or a regular database depending on the step above
    3. Set custom values for various settings. A template is provided which corresponds to a dev environment, that does not allow outside access
  3. Check if all essential settings have been provided by the custom settings
  4. Load project database connections by either:
    1. If it is a test command: Crawling through fixtures to check which projects are required and add SQLite database connections for all found
    2. If not: Check the default database for connection details to the project databases and add them

This process should have the desired behaviour for all four combinations of dev/prod environments and test/live command. I don’t see how it could happen that production data ends up in tests.