Single Model.py with Multiple Classes and Multiple Databases

[arijitArusan]

I am working on file transformation workflows and the whole project is based on Django. How the project works is , each django app is a new workflow , and once a files comes in it will invoke the workflows one after the another. My question is how to connect multiple databases from single model.py. Below is what I am trying to achieve

The Main Project Structure

Kappa
  |--Kappa
  |--FileTransformSAP
  |--FileTransformARIBA
  |--FileTransformWorkday
  .....
  ..... 

Each of the Apps have a similar Structure listed below.

FileTransformSAP
   |--__init__.py
   |--admin.py
   |--apps.py
   |--models.py
   |--tests.py
   |--views.py

I have three Databases Namely, Landing, Staging and Final

What I am trying to achieve is the model.py file will have three different classes namely, _landing, _staging and _final and this three classes should be able to interact with three different databases and mentioned above.

I am already using routers.py to segregate databases for auth and application, but I am getting stuck when single model.py containing various model class needs to be applied to different databases

Sample Model.py

class <anything>_landing(models.Model):
    definition
 
    class Meta:
        app_label = '<anything>_landing'

class <anything>_staging(models.Model):
    definition
    
    class Meta:
        app_label = '<anything>_staging'

class <anything>_final(models.Model):
    definition
    
    class Meta:
        app_label = '<anything>_final'

Along with this created the following routes.py

class LandingRouter:
    route_app_labels = {'<anything>_landing'}

    def db_for_read(self, model, **hints):
        if model._meta.app_label in self.route_app_labels:
            return "landing"
        return None

    def db_for_write(self, model, **hints):
        if model._meta.app_label in self.route_app_labels:
            return "landing"
        return None

    def allow_relation(self, obj1, obj2, **hints):
        if (
            obj1._meta.app_label in self.route_app_labels
            or obj2._meta.app_label in self.route_app_labels
        ):
            return True
        return None

    def allow_migrate(self, db, app_label, model_name=None, **hints):
        if app_label in self.route_app_labels:
            return db == "landing"
        return None

class StagingRouter:

    route_app_labels = {'<anything>_staging'}

    def db_for_read(self, model, **hints):
        if model._meta.app_label in self.route_app_labels:
            return "staging"
        return None

    def db_for_write(self, model, **hints):
        if model._meta.app_label in self.route_app_labels:
            return "staging"
        return None

    def allow_relation(self, obj1, obj2, **hints):
        if (
            obj1._meta.app_label in self.route_app_labels
            or obj2._meta.app_label in self.route_app_labels
        ):
            return True
        return None

    def allow_migrate(self, db, app_label, model_name=None, **hints):
        if app_label in self.route_app_labels:
            return db == "staging"
        return None

Similary Router for Final is also created

And here is the snippet from settings.py

DATABASES = {
   'default':{},
   'landing': {
       'ENGINE': 'django.db.backends.postgresql',
       'NAME': 'landing',
       'USER': 'postgres',
       'PASSWORD': 'admin',
       'HOST': 'localhost',
       'PORT': '5432',
   },
   'staging': {
       'ENGINE': 'django.db.backends.postgresql',
       'NAME': 'staging',
       'USER': 'postgres',
       'PASSWORD': 'admin',
       'HOST': 'localhost',
       'PORT': '5432',
   },
   'final': {
       'ENGINE': 'django.db.backends.postgresql',
       'NAME': 'final',
       'USER': 'postgres',
       'PASSWORD': 'admin',
       'HOST': 'localhost',
       'PORT': '5432',
   }
}

But this is not generating any kind of migrations. Can any one tell what approach I need to take for the situation explained

[KenWhitesell]

Notice how your router tests receive either a model or an instance as a parameter. In those tests, you’re currently using the <object>._meta.app_label as the data you’re examining within those tests to determine the database to use.

However, you’re not required or limited to only checking the app_label within your tests.

You can test any attribute of the <object> (model class or instance). This means you could write a test that checks attributes such as <object>._meta.model_name. What you test for in a router is entirely up to you.

[arijitArusan]

Can you please point me the exact modification I need to make in the code… as what I understood is if app name and ‘app_label’ Django wont do the Migrations, even if i explicitly specify it in the routers.py

[KenWhitesell]

If you’re looking to run migrations on the different databases, you would use the --database option on the migrate command.

[arijitArusan]

This is How my ‘model.py’ for the app named ‘balance_payments’ look

class payments_landing(models.Model):
    Series_reference = models.CharField(max_length=255)
    Period = models.DecimalField(max_digits=10, decimal_places=3)
    def __str__(self):
        return self.Series_reference
    class Meta:
        app_label = 'payments_landing'

class payments_staging(models.Model):
    Series_reference = models.CharField(max_length=255)
    Period = models.DecimalField(max_digits=10, decimal_places=3)
    def __str__(self):
        return self.Series_reference
    class Meta:
        app_label = 'payments_staging'

class payments_final(models.Model):
    Series_reference = models.CharField(max_length=255)
    Period = models.DecimalField(max_digits=10, decimal_places=3)
    def __str__(self):
        return self.Series_reference
    class Meta:
        app_label = 'payments_final'

and i have already given how the databases look in settings.py

now if i run the command ‘python manage.py makemigrations balance_payments’

it shows

No changes detected in app 'balance_payments

and if i run migrate directly like ‘python manage.py migrate database==landing’

CommandError: No installed app with label 'database==landing''

This is the router.py for landing

class LandingRouter:
  
    route_app_labels = {'payment_landing'}

    def db_for_read(self, model, **hints):
       
        if model._meta.app_label in self.route_app_labels:
            return "landing"
        return None

    def db_for_write(self, model, **hints):
    
        if model._meta.app_label in self.route_app_labels:
            return "landing"
        return None

    def allow_relation(self, obj1, obj2, **hints):
      
        if (
            obj1._meta.app_label in self.route_app_labels
            or obj2._meta.app_label in self.route_app_labels
        ):
            return True
        return None

    def allow_migrate(self, db, app_label, model_name=None, **hints):
     
        if app_label in self.route_app_labels:
            return db == "landing"
        return None

and it has been reference properly in settings.py as

DATABASE_ROUTERS = ['Kappa.routers.landing_router.LandingRouter']

can you please let me know where is it rerroring out

[KenWhitesell]

@arijitArusan

i run migrate directly like ‘python manage.py migrate database==landing’

That’s not the right syntax for that command. Please review the docs.

[arijitArusan]

Sorry My Mistake

i checked the command and ran it

python manage.py migrate --database=landing

It didnot migrate any thing, it also didnt generate any migrations the only table it created is migrations

I am not sure where am i going wrong…

[KenWhitesell]

You have:

@arijitArusan

route_app_labels = {'payment_landing'}

Is that the name of your app?

[arijitArusan]

This is not the Name of my app. That is where my question lies.
The App name is 'balance_payments'. The model.py in the app has three model classes class payments_landing(models.Model), class payments_staging(models.Model),class payments_final(models.Model). I want ro migrate this three classes to three different databases namely landing,staging,final . I thought of doing the same using route_app_labels but that wont work as the app name and the label name needs to be the same. So i am looking for another approach that can be used to achieve the same.

[KenWhitesell]

And that gets back to changing your router to work with the model names instead of the app name.

Your migrations aren’t going to work until your router is coded correctly to rout the requests to the right database.

[arijitArusan]

Then how do someone migrate 3 different models in the same models.py file to different databases. I didnt see any approach mentioned in the official documentation.

[KenWhitesell]

You create the router first. Once the router is created, then migrate will do what needs to be done.

[arijitArusan]

The router is migrating the 3 models in the model.py to the same database. not 3 different database

This is when i am setting the app label to the app name. What is the change i need to make to migrate 3 models to 3 different db

[KenWhitesell]

You need to create a router that routes each table to the appropriate database.

I outlined what you can do for that above at Single Model.py with Multiple Classes and Multiple Databases - #2 by KenWhitesell

[arijitArusan]

I tried setting the other attribute model_name in the model class

class Meta:
    app_label = 'balance_payments'
    model_name = 'payments_final'

but when i am trying to run the migrations it is giving the following error TypeError: 'class Meta' got invalid attribute(s): model_name

[arijitArusan]

There is a small change i made

class Meta:
        app_label = 'balance_payments'
        verbose_name = 'payments_final'

I am using the verbose_name in Meta

and there is also a change made in all the routers

class LandingRouter:
  
    route_app_labels = {'balance_payments'}
    route_app_name={'payments_landing'}

    def db_for_read(self, model, **hints):
       
        if model._meta.verbose_name in self.route_app_name:
            return "landing"
        return None

    def db_for_write(self, model, **hints):
     
        if model._meta.verbose_name in self.route_app_name:
            return "landing"
        return None

    def allow_migrate(self, db, app_label, model_name=None, **hints):
        print(db)
        if app_label in self.route_app_labels:
            if db=="landing":
                return db == "landing"
        return None

Now when i am running migrate with database name it is making the all the 3 model tabels in all the 3 databases