Feature idea: makemigrations cli flag --data to get a data migration stub

[ckirby]

For data migrations the docs say to use the --empty flag which creates a bare bones migration file. For most people the next step is probably to open up the docs and remind themselves of the method signature for forward and backwards migrations, the RunPython operation, or even the order of (forward, reverse) in the operations. I know that is what I have to do.

I propose, and am happy to implement, a makemigrations --data flag that provides a more fully stubbed out file for data migrations use. Really just taking what is already in the docs for data migrations and make it available in the management command.

Something like

from django.db import migrations

def forward_func(apps, schema_editor):
    # We can't import the model directly as it may be a newer
    # version than this migration expects. We use the historical version.
    # YourModel = apps.get_model('yourappname', 'YourModel')
    raise NotImplementedError()

def reverse_func(apps, schema_editor):
   raise migrations.exceptions.IrreversibleError()

class Migration(migrations.Migration):

    dependencies = [
        ('yourappname', 'your_most_recent_migration'),
    ]

    operations = [
        migrations.RunPython(forward_func, reverse_func),
    ]

[adamchainz]

I like the idea but would like to consider expanding it to other operations. Not all data migrations use RunPython. If you can use RunSQL, it’s faster because it avoids shuffling data to/from the DB.

Perhaps there could be some kind of “template” option that adds extra operations and associated model-level code to a migration.

By the way, one of the suggested GSoC projects is to extend migration generation to allow hooking into the generation process to detect custom operations: SummerOfCode2023 – Django . “Detecting” a RunPython operation could be implemented via this mechanism, if it existed.

@ckirby

def reverse_func(apps, schema_editor):
   raise migrations.exceptions.IrreversibleError()

There’s no need for this?

The reverse_code argument is called when unapplying migrations. This callable should undo what is done in the code callable so that the migration is reversible. If reverse_code is None (the default), the RunPython operation is irreversible.

[ckirby]

@adamchainz

I like the idea but would like to consider expanding it to other operations. Not all data migrations use RunPython. If you can use RunSQL, it’s faster because it avoids shuffling data to/from the DB.

For sure - could be a single makemigration cli argument with alternate implementations commented out in the template. Alternatively it could ask a few questions to bootstrap the process: I’m thinking:

• RunSQL or RunPython?
• Reverse Function? (yes, no, noop)
• Others?

Perhaps there could be some kind of “template” option that adds extra operations and associated model-level code to a migration.

Not sure I follow this completely. Are you suggesting prompt them for the models that will be used in the migration and code gen the get_model call?

By the way, one of the suggested GSoC projects is to extend migration generation to allow hooking into the generation process to detect custom operations: SummerOfCode2023 – Django . “Detecting” a RunPython operation could be implemented via this mechanism, if it existed.

I don’t fully grok the GSoC proposal. Can’t really speak to the feasibility of using it

@ckirby

def reverse_func(apps, schema_editor):
   raise migrations.exceptions.IrreversibleError()

There’s no need for this?

The reverse_code argument is called when unapplying migrations. This callable should undo what is done in the code callable so that the migration is reversible. If reverse_code is None (the default), the RunPython operation is irreversible.

No explicit need, no, but my aim with this proposal is to effectively have all the documented funcionality in the generated template. If this goes the direction of the code gen prompt above then a question about including a reverse function satisfies the same goal.

[adamchainz]

@ckirby

Not sure I follow this completely. Are you suggesting prompt them for the models that will be used in the migration and code gen the get_model call?

Sorry that was a typo/braino. I meant “module-level code”, like func / reverse_func.

@ckirby

I don’t fully grok the GSoC proposal. Can’t really speak to the feasibility of using it

Basically the idea is for makemigrations to be extensible so that packages can add extra “detected” operations to add to migrations. If this hook existed, you could potentially have a “RunPython detector” that is triggered when extra args are passed to makemigrations.

If there are various ideas about prompting, other kinds of templates, it could be better to start by making a package with an extended makemigrations. This woul dallow for faster experimentation.