Dynamically add a field to a model form

I have a model with a JSON field. I want to use the JSON field to store different language versions of text. So the JSON field would contain something like this:
{‘en-us’:‘Sample text’, ‘de-de’:‘Beispiel Text’}
For each pair of language and text, I want to create a pair of form fields. The first one will be non-editable and displaying the language, the second one will be a standard text field to insert the translation for that particular language. These form fields cannot be created based on the model, instead I have to create them dynamically during the initialization of the form. However, I’m not able to add a form field to a model form at all.
I tried this:

class EulogyTextForm(f.ModelForm):
    class Meta:
        model = EulogyText
        exclude = []

    def __init__(self, *args, **kwargs):
        super(EulogyTextForm, self).__init__(*args, **kwargs)

        self.fields['bla'] = f.CharField()
        self.fields['blub'] = f.IntegerField()

        # for i in range(len(settings.MULTITEXT_LANGUAGES)):
        #     field_name = 'language%s' % (i, )
        #     self.fields[field_name] = f.CharField(disabled=True, initial=settings.MULTITEXT_CHOICES[i][0])
        #     field_name += '_text'
        #     self.fields[field_name] = f.CharField()

What I commented out would be the code for actually generating the fields dynamically. That didn’t work, so I tried to just add a field to see if that would work. I’ve already read through many blogposts, forum entries, etc… All of the examples used self.fields[‘name’] to add a field. However, it doesn’t change the outcome of the form at all. I am displaying the form in the django admin, where I set the admin to explicitly use EulogyTextForm.
What am I doing wrong?

I implemented a TestView so that I could see my form outside of the admin. It turns out that all the fields are being added correctly, however the admin does not display them.

I’m guessing that the admin is caching an instance of the form when the class is registered. (I don’t know this, I’m just guessing this as a result of the symptoms you’ve described.)

Anyway, rather than trying to implement something in the __init__ method, you might try building your form dynamically in the ModelAdmin.get_form() method. (No idea if this will do you any good. I have no direct knowledge here, call it a somewhat-partially-educated guess.)

Ken

I’m guessing too that I’m failing to register the form correctly. I already tried a variation where I would assign the form in the get_form() method, but so far I had no success. I’ll keep on digging.

By the way, happy birthday to you.

It’s incredibly frustrating, as the manual at several occasions hints at a logic that is happening in the background, however it is never explained anywhere. So I went to django.contrib.admin.options and looked at get_form(). That didn’t help either. This is what I have so far:

def get_form(self, request, obj=None, change=False, **kwargs):
    # if 'fields' in kwargs:
    #     fields = kwargs.pop('fields')
    # else:
    #     fields = flatten_fieldsets(self.get_fieldsets(request, obj))
    fields = []
    for i in range(len(settings.MULTITEXT_LANGUAGES)):
        field_name = 'language%s' % (i, )
        fields.append(field_name)
        field_name += '_text'
        fields.append(field_name)

    excluded = self.get_exclude(request, obj)
    exclude = [] if excluded is None else list(excluded)
    readonly_fields = self.get_readonly_fields(request, obj)
    exclude.extend(readonly_fields)
    # Exclude all fields if it's a change form and the user doesn't have
    # the change permission.
    if change and hasattr(request, 'user') and not self.has_change_permission(request, obj):
        exclude.extend(fields)
    if excluded is None and hasattr(self.form, '_meta') and self.form._meta.exclude:
        # Take the custom ModelForm's Meta.exclude into account only if the
        # ModelAdmin doesn't define its own.
        exclude.extend(self.form._meta.exclude)
    # if exclude is an empty list we pass None to be consistent with the
    # default on modelform_factory
    exclude = exclude or None

    # Remove declared form fields which are in readonly_fields.
    new_attrs = dict.fromkeys(f for f in readonly_fields if f in self.form.declared_fields)
    form = type(self.form.__name__, (self.form,), new_attrs)

    defaults = {
        'form': form,
        'fields': fields,
        'exclude': exclude,
        'formfield_callback': partial(self.formfield_for_dbfield, request=request),
        **kwargs,
    }

    if defaults['fields'] is None and not modelform_defines_fields(defaults['form']):
        defaults['fields'] = forms.ALL_FIELDS

    try:
        return modelform_factory(self.model, **defaults)
    except FieldError as e:
        raise FieldError(
            '%s. Check fields/fieldsets/exclude attributes of class %s.'
            % (e, self.__class__.__name__)
        )

Interestingly enough, Django no longer complains about unknown fields, but it doesn’t display them either.

Are you willing to share your admin.py file with the model admin class definition?

Sure. I’m posting everything. The definition of the actual admin is just three lines after the imports. After the definition are some fruitless attempts at making get_form() work in my favor.

from django.contrib import admin
from django.contrib.admin.options import flatten_fieldsets, partial, forms, modelform_defines_fields, modelform_factory
from django.core.exceptions import FieldError
from django.utils.translation import ugettext as _
from contentmanager.models import ListedTextblock, ListedHeadline, EulogyText
from contentmanager.forms import EulogyTextForm, get_fields_list
from website import settings


class EulogyTextAdmin(admin.ModelAdmin):
    exclude = []
    form = EulogyTextForm

    def get_form(self, request, obj=None, change=False, **kwargs):
        # fields = get_fields_list()

        exclude = []
        readonly_fields = []
        if 'fields' in kwargs:
            fields = kwargs.pop('fields')
        else:
            fields = flatten_fieldsets(self.get_fieldsets(request, obj))

        # excluded = self.get_exclude(request, obj)
        # exclude = [] if excluded is None else list(excluded)
        # readonly_fields = self.get_readonly_fields(request, obj)
        # exclude.extend(readonly_fields)
        # Exclude all fields if it's a change form and the user doesn't have
        # the change permission.
        if change and hasattr(request, 'user') and not self.has_change_permission(request, obj):
            exclude.extend(fields)
        # if excluded is None and hasattr(self.form, '_meta') and self.form._meta.exclude:
        #     # Take the custom ModelForm's Meta.exclude into account only if the
        #     # ModelAdmin doesn't define its own.
        #     exclude.extend(self.form._meta.exclude)
        # if exclude is an empty list we pass None to be consistent with the
        # default on modelform_factory
        exclude = exclude or None

        # Remove declared form fields which are in readonly_fields.
        new_attrs = dict.fromkeys(f for f in readonly_fields if f in self.form.declared_fields)
        # form = type(self.form.__name__, (self.form,), new_attrs)
        form = EulogyTextForm

        defaults = {
            'form': form,
            'fields': fields,
            'exclude': exclude,
            'formfield_callback': partial(self.formfield_for_dbfield, request=request),
            **kwargs,
        }

        # if defaults['fields'] is None and not modelform_defines_fields(defaults['form']):
        #     defaults['fields'] = forms.ALL_FIELDS

        try:
            return modelform_factory(self.model, EulogyTextForm, fields=fields)
        except FieldError as e:
            raise FieldError(
                '%s. Check fields/fieldsets/exclude attributes of class %s.'
                % (e, self.__class__.__name__)
            )
    #
    # # def get_form(self, request, obj=None, **kwargs):
    # #     kwargs['form'] = EulogyTextForm
    # #
    # #     return super(EulogyTextAdmin, self).get_form(request, obj, **kwargs)
    #
    # # def get_fields(self, request, obj=None):
    # #     fields = super(EulogyTextAdmin, self).get_fields(request, obj)
    # #
    # #     for i in range(len(settings.MULTITEXT_LANGUAGES)):
    # #         field_name = 'language%s' % (i, )
    # #         fields.append(field_name)
    # #         field_name += '_text'
    # #         fields.append(field_name)
    # #
    # #     return fields
    #
    # def get_fieldsets(self, request, obj=None):
    #     fieldsets = super(EulogyTextAdmin, self).get_fieldsets(request, obj)
    #     new_fieldsets = list(fieldsets)
    #     fields = []
    #
    #     for i in range(len(settings.MULTITEXT_LANGUAGES)):
    #         field_name = 'language%s' % (i, )
    #         fields.append(field_name)
    #         field_name += '_text'
    #         fields.append(field_name)
    #
    #     new_fieldsets.append([_('Languages'), {'fields': fields}])
    #     return new_fieldsets
    # def get_form(self, request, obj=None, **kwargs):
    #     pass


# stand alone
class ListedHeadlineAdmin(admin.ModelAdmin):
    readonly_fields = ('cr_date',)

    list_display = ['level']
    search_fields = ['level']


class ListedTextblockAdmin(admin.ModelAdmin):
    readonly_fields = ('cr_date',)

    # list_display = ['level']
    # search_fields = ['level']


# Adminsites for regular models
admin.site.register(ListedHeadline, ListedHeadlineAdmin)
admin.site.register(ListedTextblock, ListedTextblockAdmin)
admin.site.register(EulogyText, EulogyTextAdmin)

I guess I’d start with a simpler attempt, just to verify that this works the way I would expect it to work:

class EulogyTextAdmin(admin.ModelAdmin):
 
    def get_form(self, request, obj=None, change=False, **kwargs):
        my_form = EulogyTextForm()
        my_form.fields['extrafield'] = forms.CharField()
        return my_form

I’d start with something this simple just to verify that I could add an additional field to the form dynamically - incrementally adding features and testing at each step.

The fact that you’re using modelform_factory would lead me to believe that it’s going to generate a model form based only upon the fields in the model. What I think you want to do is create your form, then modify it by adding your fields, and returning the result. In a way, I think you might be making it more difficult than it needs to be.

1 Like

I appreciate you taking the time to help me with this.

So what I posted here is the result of already having tried a bunch of things:

  • I explicitly declared a ModelForm for my use case. In the declaration I added a singe form field that is unrelated to the model. It works.
  • I tried the example that you have given: use get_form() to add a form field. It leads me to an “unexpeted fields” error.
  • I tried to generate the additional fields in the init of my form. Gives me “unexpected fields” error.

The reason I tried the modelform_factory method is that I was hoping that it would create the full form including the dynamically added formfields and then attach it to the model. However it doesn’t work that way.

So, I’ve tried a different approach alltogether. I get the feeling that forms are just not designed to be dynamic. My starting problem: I have a model with a JSON field containig key-value-pairs. I want each key-value-pair to have it’s own set of form field. The word set already gives it away, I turned to formsets. Instead of trying to have a form where I dynamically add in fields, I created a from with just two fields: key and value. The whole part where things get dynamic is now being handled by formsets. And I think, this is how it’s intended. I will just add those formsets to the admin and then add some logic for the additional formsets data to be turned into a dictionary to pass on to the JSONField.
Since the initial values of the formsets are determined by the value of a field on the actual model, there is still some work left to retrieve those and pass them on to the formset.
Additionally I created a custom template to include the Formsets. So this seems to be a way to have formsets with dynamically generated fields in the django admin.

1 Like

It may very well be that dynamic forms are one of those areas where the admin is just not designed to be extended. Sounds like you’ve got a good workable solution though.

I’m playing around with a different idea though, just from my own curiosity. I’ll let you know if I figure something out.

I would appreciate any input.

Ok, as someone who has been known to abuse the admin on more than one occasion, I did some digging. I can’t really summarize everything here, because a lot of it is still a little hazy to me, but…

in admin.py

class MyAdmin(admin.ModelAdmin):
    def get_form(self, request, obj=None, change=False, **kwargs):
        return MyForm

in forms.py

class MyForm(forms.ModelForm):
    class Meta:
        model = MyModel
        exclude = []

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        # The following dynamically add three fields to the model form
        for x in ['e1', 'e2', 'e3']:
            self.base_fields[x] = forms.CharField()
            self.fields[x] = forms.CharField()

    def save(self, *args, **kwargs):
        # The following demonstrates that the three extra fields are passed into cleaned_data
        print(self.cleaned_data)
        return super().save(*args, **kwargs)

Use as your own risk. No warranty, guarantee, or assurance applies to any of the above. All disclaimers apply. If this causes your computer to melt, don’t blame me.

From the docs at https://docs.djangoproject.com/en/3.0/ref/forms/api/:

Beware not to alter the base_fields attribute because this modification will influence all subsequent ContactForm instances within the same Python process.

so this might not be as dynamic as desired. (Of course, I’m one of those that believes the admin facility should be restricted to as few people as possible. If you’re using this in any situation where you might be expecting multiple people to be doing this at the same time, then I would argue you’re misusing the admin. In the normal/routine case, the admin wouldn’t be any part of your user-facing site, and so you could perform these dynamic modifications from within your view.)

Ken

Interesting. I had tried filling the base_fields, but I never tried filling both the fields and the base_fields. I do not want to change the form dynamically after initiation. I have made an addition to my settings.py. The idea is that you add all the languages that you want to be availabe to a dictionary and then the form builds itself based on that information. So I think your solution might just work for me. I’ll try it out tomorrow. Thanks for your digging!

It’s still rather ugly, but it works now. Thanks again for your help!

1 Like

I had a similar requirement for using a JSONField as a field to store various versions of languages. These versions are based on LANGUAGES setting of Django. Combining all of the above answers & approaches, I made it working in this way. Hope it helps someone!

Requirements:

  • It shouldn’t alter schema
  • When LANGUAGES setting is updated (e.g. one more language is added/removed), a corresponding field must be added/removed automatically.
  • Default (first) language field must be required.

utils.py

from django.conf import settings

def dynamic_admin_fields(form: forms.ModelForm, field_name: str) -> dict:
    fields = {}

    for index, (code, lang) in enumerate(settings.LANGUAGES):
        init_value = form.initial.get(field_name, {}).get(code, "")
        form_value = forms.CharField(
            label=f"{lang} {field_name.title()}",
            initial=init_value,
            help_text=f"{field_name.title()} in {code} ({lang})",
            required=True if index == 0 else False, # First Language must be required
        )
        fields.update({f"{field_name}_{code}": form_value})
    return fields

forms.py

from django import forms
from django.conf import settings

from .utils import dynamic_admin_fields
from .models import Article

class ArticleForm(forms.ModelForm):

    dynamic_root_field = "title"
    def __init__(self, *args, **kwargs) -> None:
        super(ArticleForm, self).__init__(*args, **kwargs)

        custom_fields = dynamic_admin_fields(self, self.dynamic_root_field)
        self.base_fields.update(custom_fields)
        self.fields.update(custom_fields)

        # Disable title field in admin
        self.base_fields.get(self.dynamic_root_field).disabled = True
        self.fields.get(self.dynamic_root_field).disabled = True

    class Meta:
        model = Article
        fields = "__all__"

    def save(self, commit):
        for code, _ in settings.LANGUAGES:
            val = self.cleaned_data.get(f"{self.dynamic_root_field}_{code}", "")
            self.cleaned_data[self.dynamic_root_field].update({code: val})
        return super().save(commit)

admin.py

from django.contrib import admin

from .forms import ArticleForm
from .models import Article

class ArticleAdmin(admin.ModelAdmin):
    list_display = ("title", "published", "body", "created")

    def get_form(self, request, obj=None, change=False, **kwargs):
        return ArticleForm

admin.site.register(Article, ArticleAdmin)

This might be a bit late, but I’ve been struggling with the same.

Considering the through model can be automatically added (that is, its non-foreignkey fields have defaults), my models look like this:

from django.db.models import CharField
from django.db.models.base import Model
from django.db.models.fields import PositiveSmallIntegerField
from django.db.models.indexes import Index
from django_stubs_ext.db.models import TypedModelMeta
from ktools.django.utils.translation import gettext_safelazy as _


class NewsletterCategory(Model):
    'Storing categories for the `Newsletter`s.'
    name = CharField(verbose_name=_('name'), max_length=100)
    description = CharField(verbose_name=_('description'), max_length=255)
    sort_value = PositiveSmallIntegerField(
        verbose_name=_('Sort value'), default=10, help_text=_(
            'The value by which this item will get sorted in the list of ' +
            'newsletter categories. Lower is better.'))

    class Meta(TypedModelMeta):
        verbose_name = _('Newsletter category')
        verbose_name_plural = _('Newsletter categories')
        ordering = ('sort_value', 'name')
        indexes = [
            Index(fields=['sort_value', 'name'], name='s')
        ]

    def __str__(self):
        return self.name

class User(AbstractUser):
    slug = AutoSlugField(
        verbose_name=_('Slug of the user'), max_length=_username_max_length,
        unique=True, populate_from='username', slugify_function=slugify)
    phone = PhoneNumberField(verbose_name=_('Phone number'), blank=True)
    newsletter_categories = ManyToManyField(
        to=NewsletterCategory, through='UsersettingsToNewslettercategories',
        verbose_name=_US_NC_VERBOSE_PLURAL, blank=True)

    class Meta(AbstractUser.Meta, TypedModelMeta):
        abstract = False
        ordering = ('-date_joined',)


class UsersettingsToNewslettercategories(Model):
    user = ForeignKey(to=User, on_delete=CASCADE)
    newslettercategory = ForeignKey(
        verbose_name=NewsletterCategory._meta.verbose_name,
        to=NewsletterCategory, on_delete=CASCADE)
    date_changed = DateTimeField(
        verbose_name=_('Date changed'), default=now, editable=False)

    class Meta(TypedModelMeta):

        verbose_name = _US_NC_VERBOSE
        verbose_name_plural = _US_NC_VERBOSE_PLURAL
        constraints = [
            UniqueConstraint(
                fields=['user', 'newslettercategory'],
                name='unique-constraint')
        ]

    def __str__(self):
        return f'{self.user} -> {self.newslettercategory}'

Now, I’ve had to dive into metaclasses and stuff to get the original filter_horizontal (or filter_vertical) to display right, circumventing django-admin’s checks where it will refuse to display that widget for a ManyToManyField that has a through specified. Brace yourselves, here comes my admin.py that solves the problem:

from __future__ import annotations

from typing import Any, ClassVar

from django.contrib.admin.options import ModelAdmin
from django.contrib.admin.widgets import FilteredSelectMultiple
from django.forms.fields import CharField, MultipleChoiceField
from django.forms.models import ModelForm, ModelFormMetaclass
from django.http.request import HttpRequest

from .models import User

# Metaclass typing is unavailable with static type checking, see:
# https://github.com/microsoft/pyright/issues/7311#issuecomment-1956995047

class MyUserAdminFormMetaclass(ModelFormMetaclass):
    base_fields: ClassVar[dict[str, CharField | MultipleChoiceField]]

    def __new__(
        mcs: type[MyUserAdminFormMetaclass], name: str,
        bases: tuple[type, ...], attrs: dict[str, Any]
    ) -> type[MyUserAdminFormMetaclass]:
        new_class = super().__new__(mcs, name, bases, attrs)
        new_class.base_fields['hacked_newsletter_categories'] = \
            new_class.base_fields['newsletter_categories']
        del new_class.base_fields['newsletter_categories']
        return new_class


class MyUserAdminForm(ModelForm, metaclass=MyUserAdminFormMetaclass):

    class Meta(object):
        model = User
        fields = (
            'username', 'last_name', 'first_name', 'is_active', 'email',
            'newsletter_categories')
        widgets = dict(newsletter_categories=FilteredSelectMultiple(
            verbose_name=User.newsletter_categories.field.verbose_name,
            is_stacked=True))

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        if 'newsletter_categories' in self.initial:
            initial = dict(self.initial)
            initial['hacked_newsletter_categories'] = \
                initial['newsletter_categories']
            del initial['newsletter_categories']
            self.initial = initial

    def full_clean(self):
        super().full_clean()
        if self.is_bound:
            self.cleaned_data['newsletter_categories'] = \
                self.cleaned_data['hacked_newsletter_categories']
            del self.cleaned_data['hacked_newsletter_categories']


class UserAdmin(ModelAdmin):
    model = User
    list_filter = ('is_active', 'date_joined', 'first_name')
    list_display = (
        'username', 'is_active', 'date_joined', 'last_name', 'first_name')
    fields = (
        'username', ('last_name', 'first_name'), 'is_active', 'email',
        'hacked_newsletter_categories')
    date_hierarchy = 'date_joined'
    autocomplete_fields = ['newsletter_categories']

    def get_form(
        self, request: HttpRequest, obj: User | None = None,
        change: bool = False, **kwargs
    ):
        return MyUserAdminForm

I spent a couple days in getting this working right. It is crazy I know, but here it is for people coming after me, pondering the same problem.

Feel free to (ab)use it in any way you like.