Can the migration system be extended by third-party apps?

Hey folks, does the migration autodetector support being extended by third-party apps? I would like to have a custom model field which includes an extra operation after the AddField operation in the migration.

The ultimate goal is to be able to generate a new operation which would emit the following SQL:

SECURITY LABEL FOR anon ON COLUMN myapp_mymodel.name
IS 'MASKED WITH FUNCTION anon.dummy_last_name()';

For example, for a model such as:

class MyModel():
    name = SensitiveTextField(anon_func="dummy_last_name")

It’d generate a migration with:

operations = [
    AddField(),
    ApplySecurityLabel("mymodel", "name", anon_func="dummy_last_name"),
]

I suppose another way this could be implemented would be to hijack the AddField() operation to generate other SQL.

1 Like

@bmispelon pointed out that indexes and constraints may allow for a work-around. That seems to have worked.

from django.db import models
from django.db.backends.ddl_references import Table, Statement

class SecurityLabel(models.Index):
    def __init__(self, *args, anon_function, fields=(), **kwargs):
        self.anon_function = anon_function
        if len(fields) != 1:
            raise ValueError(f"{self.__class__.__name__} must be used with exactly one field.")
        super().__init__(*args, fields=fields, **kwargs)

    def _remove_security_label(self):
        return "SECURITY LABEL FOR anon ON COLUMN %(table)s.%(column)s IS NULL"

    def _get_security_label(self):
        return "SECURITY LABEL FOR anon ON COLUMN %(table)s.%(column)s IS 'MASKED WITH FUNCTION anon.dummy_last_name()'"

    def create_sql(self, model, schema_editor, using="", **kwargs):
        return Statement(
            self._get_security_label(),
            table=Table(model._meta.db_table, schema_editor.quote_name),
            column=schema_editor.quote_name(model._meta.get_field(self.fields[0]).column),
            anon_function=schema_editor.quote_name(self.anon_function),
        )

    def remove_sql(self, model, schema_editor, **kwargs):
        return Statement(
            self._remove_security_label(),
            table=Table(model._meta.db_table, schema_editor.quote_name),
            column=schema_editor.quote_name(model._meta.get_field(self.fields[0]).column),
        )

    def deconstruct(self):
        (path, expressions, kwargs) = super().deconstruct()
        kwargs["anon_function"] = self.anon_function
        return path, expressions, kwargs

Then the model would be:

from django.db import models
from anon.fields import SecurityLabel

class MyModel(models.Model):
    field = models.TextField()

    class Meta:
        indexes = [
            SecurityLabel(fields=("field", ), anon_function="dummy_last_name")
        ]

The initial migration generates the following SQL:

BEGIN;
--
-- Create model MyModel
--
CREATE TABLE "home_mymodel" ("id" integer NOT NULL PRIMARY KEY GENERATED BY DEFAULT AS IDENTITY, "field" text NOT NULL);
SECURITY LABEL FOR anon ON COLUMN "home_mymodel"."field" IS 'MASKED WITH FUNCTION anon.dummy_last_name()';
COMMIT;

Then changing the anon_function to dummy_first_name:

BEGIN;
--
-- Remove index home_mymode_field_39b899_idx from mymodel
--
SECURITY LABEL FOR anon ON COLUMN "home_mymodel"."field" IS NULL;
--
-- Create index home_mymode_field_39b899_idx on field(s) field of model mymodel
--
SECURITY LABEL FOR anon ON COLUMN "home_mymodel"."field" IS 'MASKED WITH FUNCTION anon.dummy_last_name()';
COMMIT;
2 Likes

django.contrib.contenttypes does something like this, except it doesn’t “bake” the operations into the migration files; it just injects them into the migration plan. See inject_rename_contenttypes_operations() and also an accepted ticket for making django.contrib.auth do something similar.

1 Like

There’s some gold in that ticket discussion:

Another alternative could be to introduce a post_operation signal where the sender would be the operation class, operation the Operation instance and we could include the from_state and to_state as well.

The content types app could register a signal receiver for CreateModel using post_operation.connect(sender=CreateModel, receiver) and return [CreateContentType(...)]. The executor would then execute the collected operations. Then auth app could register a post_operation.connect(sender=CreateContentType, receiver) and return [CreatePermission] which the executor would execute as well.

That would allow us get rid of the ​nasty plan injection logic and expose a somewhat reusable interface for third party apps all that without enforcing INSTALLED_APPS ordering for now.


Could be a GSoC idea, maybe? To create the interface, and then convert contenttypes and auth to use it.

4 Likes