Optional migrations for reusable app

Using Django
1

I’m wondering if the migration framework has the capability to offer some sort of optional migration or DDL migration that could be run outside the normal migrate command, as a one off. This would I think be quite useful for reusable 3rd party apps, when a migration might depend on a setting of the reusable app.

To cite a concrete example, django-celery-results has an open issue which seems quite tricky to solve properly:

For context, the project offers to store results of Celery task runs into the database, in a TaskResult model. However, this model was created with an AutoField primary key, and the primary key can easily to overflow on system with high traffic. The issue would be solved by changing the type of the primary key to BigAutoField. However, that requires a database migration which can be unsafe to run, depending on the workload of the deployment, which is not under the control of the open source library.

I started to explore adding a setting to let us customise the type of the field, but that doesn’t provide a good way for existing deployments to migrate and change the field type.

As the package maintainers don’t control the target where it’s deployed, it’s tricky to coordinate such migration, so I was wondering if there was a way for the reusable app to provide something that could be run by whoever controls the deployment, when they’re comfortable doing so:

  1. Upgrade to the newer version with configurable field, set the configuration to remain as AutoField
  2. Later on, when they need it, change the setting to BigAutoField and run the migration/management command/script to perform the migration

The package could provide a management command as raw SQL, but that wouldn’t be database agnostic.

Is this easily doable in Django? If not, would there be an appetite for such way of running DDL changes?

1 Like