Discussion: Option for not tracking unmanaged models in migrations?

Django Internals ORM
1

I’ve been thinking a lot about unmanaged models / managed models, because I deal a lot with both regularly (probably more than the average user I think). We deal with read-only access to 3rd party db’s as well as specific views and stored procedures on db’s we own/administrate.

I’m wondering if there may not be some benefit to an option to ‘not track changes’ to particular ( and/or all) unmanaged models.

It would add more flexibility to allow if it allowed people to exclude any/all unmanaged models from migrations. Because the models are unmanaged - the question needs to be asked “Is there a time where we don’t need to be tracking changes?” I think the answer is, “Yes.”

Drawing from the django docs:

You should think of migrations as a version control system for your database schema. makemigrations is responsible for packaging up your model changes into individual migration files - analogous to commits - and migrate is responsible for applying those to your database.

With regards to models relating to dbs/tables out of your control (think 3rd party applications) do we need those changes historically? To give what I think it is a fitting analogy, we don’t track commits to 3rd party libraries (for example venv directories or node_modules are generally git ignored), so I think we should be able to determine if certain models could be ‘ignored’ from migrations.

In the cases of unmanaged models specifically here are my thoughts:

I think the goal is to get django to currrently track everything with them (similar to managed models) so migrations truly work as a ‘version control’ (see the ticket to ensure this works).

Being able to keep certain unmanaged models from being migrated would make djangos behavior more in line with the documentation and give more validity to the docs idea that migrations are analogous to commits. Similar to excluding 3rd party libraries from vcs, we should be able to exclude models (which may be related to 3rd party dbs) from migrations.

An issue was brought up by @charettes

  • What happens if we stop ‘not including migrations’ half way through?

I think we consider it in the same vein as the current option of managing/not-managing models. In these cases we specifically allow a migration for those models, should they be getting changed from one to the other.

We could potentially implement some kind of check in the autodetector.

Scenario 1 no-migrations → migrations:
In this scenario, we could ‘include a migration for that change’ (maybe ‘CreateModel’ type).

Scenario 2 migrations → no-migrations:
In this scenario, we could include a migration that is a ‘DeleteModel’ type, and then any further changes would not be tracked by migrations.

This is just opening up ideas for discussion. I’m interested to hear what others have to say/think about this.

2

Thanks for filling up this thread @hannylicious and your work on the unmanaged models tracking work.

Taking a minute to reflect about this more this morning I think that the main reason why unmanaged started being tracked in the first place is to support the case where a managed model references an unmanaged model through a foreign key. In this case where relationships cross the management boundary we must track referenced unmanaged models in order to generate the proper REFERENCES clause and allow joining against them.

I guess that this case could be solved by raising an exception when --ignore-unmanaged (or whatever opt-in mechanism is deemed appropriate) is requested but there’s a new managed model referencing an unmanaged one.

With all the edge case required to get this right and the apparent low usage of unmanaged models in migrations (it took us almost a decade to notice the problem you recently reported) I think that your usage could be qualified as uncommon and thus not meet the bar for inclusion in core particularly with the upcoming support for user defined auto-detector classes that would allow you to systematically exclude unmanaged models from to_state and thus achieve what you’re after.

3

Great points - and thank you for the continued discussion on such an admittedly unusual topic. I greatly appreciate your insights. I had completely forgotten about the idea of other models referencing an unmanaged model via FK. And that raises some more unique issues/concerns for sure.

I realize this is a fairly unique edge use case. I have (in the past when googling for this or that related to unmanaged models) ran across others wishing to do the same (exclude them from migrations). However, it was indeed very few - often spread out amongst various years. Hardly a genuine ‘interest’ in the feature I think.

That upcoming support for user defined auto-detector classes will fit the bill quite nicely for the few apps that I have which are entirely filled with unmanaged models (3rd party read-only DBs). Looking forward to it.

Here’s to the continued success and growth of one of the best frameworks around. Cheers!