Problems with multiple databases migrations and tests

Using Django
1

Hello there,

I’m currently adding another database to save models of newapp only inside it.
I want all the other applications to save their models inside the default one as before.

"newapp_db": {
    "ENGINE": "django.db.backends.postgresql",
    "NAME": env("NEWAPP_POSTGRES_DB"),
    "USER": env("NEWAPP_POSTGRES_USER"),
    "PASSWORD": env("NEWAPP_POSTGRES_PASSWORD"),
    "HOST": env("NEWAPP_POSTGRES_HOST"),
    "PORT": env("NEWAPP_POSTGRES_PORT"),
    "TEST": {"NAME": "newapptest"},
    "CONN_MAX_AGE": 60 if DEPLOYMENT != "LOCAL" else 0,
    "ATOMIC_REQUESTS": True,
},

I also added this setting:
DATABASE_ROUTERS = ["path.to.NewappRouter"]

And this is the code for the NewappRouter:

class NewappRouter:
    """
    A router to control all database operations on models in the
    newapp application.
    """

    route_app_labels = {"newapp"}

    def db_for_read(self, model, **hints):
        """
        Attempts to read newapp models go to cardtokens_db.
        """
        if model._meta.app_label in self.route_app_labels:
            return "newapp_db"
        return None

    def db_for_write(self, model, **hints):
        """
        Attempts to write newapp models go to newapp_db.
        """
        if model._meta.app_label in self.route_app_labels:
            return "newapp_db"
        return None

    def allow_relation(self, obj1, obj2, **hints):
        """
        Allow relations for newapp app only.
        """
        if (
            obj1._meta.app_label in self.route_app_labels
            and obj2._meta.app_label in self.route_app_labels
        ):
            return True
        return None

    def allow_migrate(self, db, app_label, model_name=None, **hints):
        """
        Make sure the newapp app only appear in the
        'newapp_db' database.
        """
        if app_label in self.route_app_labels:
            return db == "newapp_db"
        return False

**Problems: **

  • When I run manage.py migrate, migrations from newapp are applied to the default database. Then when I run manage.py migrate --database=newapp_db the other apps migrations get applied to the newapp_db! And both cases should be prevented by the router according to the documentation
  • When I run manage.py test I get the errors AssertionError: Database connections to 'newapp_db' are not allowed in this test. Add 'newapp_db' to path.to.TestCaseName.databases to ensure proper test isolation and silence this failure. which also should be prevented by the router!

Knowing that I inspected the databases using manage.py dbshell and found that the operations are done as desired, newapp models are saved into newapp_db and not in the default

Any ideas what might be the problem?

Thanks.

1 Like
2

Have you verified that the tables, etc are actually being created in the wrong databases?

Migrations will show the migrations as being applied, but if the router is working correctly, nothing happens.

For example, I added a “newapp” app to a system with your router, and made appropriate configuration changes. I ran makemigrations - the migration was created properly.
I then ran migrate, and it showed that the migration ran, but an inspection of the database also shows that the tables were not created.
I then ran migrate with the --database newapp_db parameter. It likewise showed that all the migrations ran, but the only tables created were in my newapp models.

1 Like
3

Thank you. I double-checked and I have the same results. It seems like Django prints the migrations anyway.

And we worked around the tests issue by adding databases = {"default", "newapp_db"} to every TestCase using a mixin:

class MultiDbMixin:
    databases = {"default", "newapp_db"}
4

The migration is “applied” whether or not the migrations actually change a database. While I don’t know all the specific reasons why it was done this way, it does makes sense to me. Migrations have a specific order that they need to be run, and so a later migration needs to know that all the previous migrations have completed. (You will find the django_migrations table in every database used for migrations - that’s where this tracking is stored.)

5

I had the same problem, then I added the following:


    def allow_relation(self, obj1, obj2, **hints):
        if obj1._meta.app_label in self.route_app_labels and obj2._meta.app_label in self.route_app_labels:
            return True

        elif self.route_app_labels not in [obj1._meta.app_label, obj2._meta.app_label]:
            return None
        return None

    def allow_migrate(self, db, app_label, model_name=None, **hints):
        if app_label in self.route_app_labels:
            return db == 'newapp_db'
        elif db == 'newapp_db':
            return False
        return None
6

What excatly mean with “relations” ? What is happening exactly on ‘def allow_relation’?

7

The allow_relation method is explained on the Multiple databases page.

1 Like