# Help with many to many formsets

**URL:** <https://forum.djangoproject.com/t/help-with-many-to-many-formsets/7328>\
**Category:** Forms & APIs\
**Created:** [April 1, 2021, 5:43pm UTC](https://forum.djangoproject.com/t/help-with-many-to-many-formsets/7328 "2021-04-01T17:43:21Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![typonaut](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.djangoproject.com/typonaut/32/2316_2.png) [@typonaut](https://forum.djangoproject.com/u/typonaut)\
**Post date:** [April 1, 2021, 5:43pm UTC](https://forum.djangoproject.com/t/help-with-many-to-many-formsets/7328/1 "2021-04-01T17:43:21Z")

</div>

I am struggling with getting a usable many to many formset working – I have it largely functional, but from a UI perspective it is unusable, because it shows a select list with 16,000 `objects` in it (ie `DirectoryLink3 object (32)` is in the list, rather than `Take away shops`).

You can imagine that getting/formatting a select with 16,000 objects, multiple times, is a little slow, not even considering the time it takes to render in a browser.

As it is the forms save/delete formsets as expected. However, in this case I do not want a select list for each form, I just want the `description` field from the `DirectoryLink3` model as a label/text (not input) and a delete checkbox for each form in the formset.

That is to say that the user won’t be able to edit the `description`, just delete the row. To add a row I have a separate autocomplete input `categories`, this uses javasccript to populate the required hidden form elements for a new row. The `categories` autocomplete seems to be working as expected, although I have yet to test it actually adding a record.

So, in breif: how to get the `description` field data without 16,000 siblings, show it as text on the form and control what ever other elements are in the object (ie do not show them).

```auto
models.py
   …
    user = models.OneToOneField(settings.AUTH_USER_MODEL, on_delete=models.CASCADE)
   …
    directory_optin = models.BooleanField(_('Directory opt-in'), default=False, db_index=True)
    directory_desc = models.CharField(_('Business description'), max_length=512, default='', blank=True, null=False, db_index=True)
    directory_classes = models.ManyToManyField('DirectoryLink3', through='directorylink_profiles' )

class DirectoryLink3(models.Model):
    id = models.BigAutoField(primary_key=True)
    profiles = models.ManyToManyField('Profile', through='DirectoryLink_profiles')
    uplink = models.CharField(max_length=16, blank=True, null=True, db_index=True)
    catid = models.CharField(max_length=16, blank=True, null=True, db_index=True)
    description = models.CharField(max_length=128, blank=True, null=True, db_index=True)
    createdate = models.DateTimeField(auto_now_add=True)
    moddate = models.DateTimeField(auto_now=True)

    class Meta:
        db_table = 'directorylink_3'

class DirectoryLink_profiles(models.Model):
	id = models.BigAutoField(primary_key=True)
	directorylink = models.ForeignKey(DirectoryLink3, null=True, on_delete=models.CASCADE)
	profile = models.ForeignKey(Profile, verbose_name=_('profile'), null=True, on_delete=models.CASCADE)

```

```auto
views.py

class DirectoryUpdateView(UpdateView):
	template_name = 'directory/edit_directory.html'
	form_class = DirectoryForm
	model = Profile
	success_url = reverse_lazy('directory:directory-edit')

        #tried this, did not seem to work
	#formset_class = inlineformset_factory(Profile, DirectoryLink_profiles, fields=('directorylink', 'profile'))
	
	#required because there is no pk or slug
	def get_object(self, queryset=None):
		return Profile.get_or_create_for_user(self.request.user)

	def form_valid(self, form):
		print('form: ', dir(form))
		if form.data['directory_desc'] == '':
			messages.warning(self.request, 'The ‘directory description’ field is empty, \
				this may not be an error, but it is more useful to complete this field.')
			#form.add_error('directory_desc', 'Hello')
		self.object = form.save()
		messages.success(self.request, 'Changes to item have been saved')
		
		context = self.get_context_data(form=form)
		formset = context['directorychoice']
		if formset.is_valid():
			response = super().form_valid(form)
			formset.instance = self.object
			formset.save()
			return response
		else:
			return super().form_invalid(form)
		
	@method_decorator(login_required)
	def dispatch(self, *args, **kwargs):
		#print('self.formset: ', self.formset)
		log_views(self)
		return super(DirectoryUpdateView, self).dispatch(*args, **kwargs)
		
	def get_context_data(self, **kwargs):
	
		context = super(DirectoryUpdateView, self).get_context_data(**kwargs)
		
		if self.request.POST:
			context['directorychoice'] = DirectoryChoice(self.request.POST, instance=self.object)
			context['directorychoice'].full_clean()
		else:
			context['directorychoice'] = DirectoryChoice(instance=self.object)
		return context

```

```auto
forms.py

class DirectoryForm(forms.ModelForm):

    directory_desc = forms.CharField(label=_('Directory description'), widget=forms.Textarea(attrs={'rows':5, 'blank': True}), required=False, help_text=_('Give a short description of your business, up to 512 characters') )
    directory_list = forms.CharField(label=_('Categories'), help_text=_('Type a few characters and select from the pop-up menu'), required=False)

    class Meta:
        model = Profile
        
        fields = (
            'directory_optin',
            'directory_desc',
            'directory_list',
        )

DirectoryChoice = forms.models.inlineformset_factory(
							Profile, 
							DirectoryLink_profiles, 
							fields = ['id', 'directorylink', 'profile'], 
							exclude = [], 
							can_delete = True,
							max_num=3,
							extra=1,
							)

```

```auto
edit_directory.html

…
{{ directorychoice.management_form }}
…
{% bootstrap_formset directorychoice layout='horizontal' %}
…

```

Sorry, that posted before I’d finished.

---

<div class="post-metadata">

**Author:** ![KenWhitesell](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.djangoproject.com/kenwhitesell/32/280_2.png) [@KenWhitesell](https://forum.djangoproject.com/u/KenWhitesell)\
**Post date:** [April 2, 2021, 2:25pm UTC](https://forum.djangoproject.com/t/help-with-many-to-many-formsets/7328/2 "2021-04-02T14:25:53Z")

</div>

inlineformset\_factory takes a named parameter `form`, which would be a ModelForm class (not instance!) for use by the factory function. You can create your form class to be whatever you want it to be - custom widgets, fields, whatever.

See [Specifying widgets to use in the inline form](https://docs.djangoproject.com/en/3.1/topics/forms/modelforms/#specifying-widgets-to-use-in-the-inline-form) and the [ModelForm factory function](https://docs.djangoproject.com/en/3.1/topics/forms/modelforms/#modelform-factory-function) docs.

---

<div class="post-metadata">

**Author:** ![typonaut](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.djangoproject.com/typonaut/32/2316_2.png) [@typonaut](https://forum.djangoproject.com/u/typonaut)\
**Post date:** [April 2, 2021, 4:37pm UTC](https://forum.djangoproject.com/t/help-with-many-to-many-formsets/7328/3 "2021-04-02T16:37:08Z")

</div>

Ken, I’m taking it that you mean this line here (and the one just below it):

```auto
			context['directorychoice'] = DirectoryChoice(self.request.POST, instance=self.object)

```

I took this from someone else’s example, which seems to work.

I believe I tried several ways to use a `form` but could not get it to work.

However, I have come to the realisation that, whatever widget I apply to a field, I am still going to end up with 16,000 of them per form (ie I’ll get 16,000 in a select, 16,000 checkboxes or 16,000 text input fields), which is absolutely no use to anyone.

With that in mind I have a plan for a solution, which I’ll post if/when I get it working.

I also realised that it isn’t the retrieval of 16,000 records that is bringing the proces to its knees (the query and the return of the data is nearly instantaneous), it’s django’s processing and formatting, plus the browser churning through around 7MB of HTML. So, important to reduce that load.

---

<div class="post-metadata">

**Author:** ![KenWhitesell](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.djangoproject.com/kenwhitesell/32/280_2.png) [@KenWhitesell](https://forum.djangoproject.com/u/KenWhitesell)\
**Post date:** [April 2, 2021, 4:43pm UTC](https://forum.djangoproject.com/t/help-with-many-to-many-formsets/7328/4 "2021-04-02T16:43:06Z")

</div>

Actually, I was referring to this line:

> [@typonaut](#):
>
> ```auto
> DirectoryChoice = forms.models.inlineformset_factory(
> Profile, 
> DirectoryLink_profiles, 
> fields = ['id', 'directorylink', 'profile'], 
> exclude = [], 
> can_delete = True,
> max_num=3,
> extra=1,
> )
> 
> ```

You can define a form class with whatever structure you desire and pass it to the factory using the `form` parameter.

```auto
DirectoryChoice = forms.models.inlineformset_factory(
    Profile, DirectoryLink_profiles, form=CustomModelForm, ...
)

```

where CustomModelForm is a Form class you create to build the form you want used in the formset.

---

<div class="post-metadata">

**Author:** ![typonaut](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.djangoproject.com/typonaut/32/2316_2.png) [@typonaut](https://forum.djangoproject.com/u/typonaut)\
**Post date:** [April 2, 2021, 4:48pm UTC](https://forum.djangoproject.com/t/help-with-many-to-many-formsets/7328/5 "2021-04-02T16:48:47Z")

</div>

Oh, I got that part. Like I wrote, I couldn’t get that to work as expected. But I’ll give it another go when I get to that part again.

The bit I was referring to was where you wrote:

> (not instance!)

But I see what you mean now.

I take it you agree that the widget type does not matter, I’m still going to get 16,000 of them?

---

<div class="post-metadata">

**Author:** ![KenWhitesell](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.djangoproject.com/kenwhitesell/32/280_2.png) [@KenWhitesell](https://forum.djangoproject.com/u/KenWhitesell)\
**Post date:** [April 2, 2021, 4:59pm UTC](https://forum.djangoproject.com/t/help-with-many-to-many-formsets/7328/6 "2021-04-02T16:59:52Z")

</div>

No, my understanding from what you wrote here:

> [@typonaut](#):
>
> As it is the forms save/delete formsets as expected. However, in this case I do not want a select list for each form, I just want the `description` field from the `DirectoryLink3` model as a label/text (not input) and a delete checkbox for each form in the formset.

So you wouldn’t necessarily use a widget - just a text field with the desired description (unless I’m getting my fields confused from your example - in which case I apologize for my confusion.).

You don’t _need_ to use any type of selection box for a field, and if it’s a non-editable field, you wouldn’t need to use a widget at all - just use the value as a label. (If you wanted it to _look_ like a widget, you could use a normal text-entry box as the widget with the value as the default value and make it disabled / non-editable.)

That’s why a custom form may be useful here - you will have full control over what’s rendered and how.

(You can also create a view to test your new form, to ensure that it’s going to render correctly, before applying it to your formset - I’ve found it helpful in the past to do that.)

Note, if you do want it to be an editable field, there are also ajax-based “auto-complete” fields that greatly reduce the size of the initial form at the expense of the ajax calls to retrieve partial results.

---

<div class="post-metadata">

**Author:** ![typonaut](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.djangoproject.com/typonaut/32/2316_2.png) [@typonaut](https://forum.djangoproject.com/u/typonaut)\
**Post date:** [April 2, 2021, 5:29pm UTC](https://forum.djangoproject.com/t/help-with-many-to-many-formsets/7328/7 "2021-04-02T17:29:11Z")

</div>

I think that it doessn’t matter if it is an imput type or a label, there are still going to be 16k instances – I looked at the queries, there is no `WHERE` clause, so it is getting everything in the table.

Ajax is the obvious solution to populating new rows, this is what I mention in my original post – this complex piece of UI is actually amongst the easiest parts of the problem to solve!

Thanks for the feedback.

---

<div class="post-metadata">

**Author:** ![KenWhitesell](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.djangoproject.com/kenwhitesell/32/280_2.png) [@KenWhitesell](https://forum.djangoproject.com/u/KenWhitesell)\
**Post date:** [April 2, 2021, 6:13pm UTC](https://forum.djangoproject.com/t/help-with-many-to-many-formsets/7328/8 "2021-04-02T18:13:45Z")

</div>

I’m curious, you’ve lost me here. 16K instances of what? Are you saying you’re generating 16K rows in your formset? (I was under the impression that you’re talking about 16K choices within a single select field, in which case changing the select field to a text box in the form is _not_ going to generate 16K elements, because you’re not going to query on the entire table - you’re retrieving a single value in your form for that box.)

---

<div class="post-metadata">

**Author:** ![typonaut](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.djangoproject.com/typonaut/32/2316_2.png) [@typonaut](https://forum.djangoproject.com/u/typonaut)\
**Post date:** [April 2, 2021, 7:59pm UTC](https://forum.djangoproject.com/t/help-with-many-to-many-formsets/7328/9 "2021-04-02T19:59:34Z")

</div>

No, you’re right. I got the `form` input in the `inlineformset_factory` working, then when the `directorylink` field is set to the `TextInput` widget the entire query to get the data for the formset changes.

My form looks like this:

```auto
class DirectoryChoices(forms.ModelForm):
	
	class Meta:
		model = DirectoryLink_profiles
		
		fields = (
			'uselesstext',
			'profile',
			'directorylink',
			)
			
		
		widgets = {
            'directorylink': forms.TextInput(attrs={'placeholder': 'Do not edit!'}),
        }

```

Following which, my only problem is getting the `description` field on the form, where the `directorylink` field is the foreign key to the `DirectoryLink3` model.

In SQL terms:

```auto
select DirectoryLink3.description from DirectoryLink_profiles
join DirectoryLink3 on DirectoryLink_profiles.directorylink = DirectoryLink3.id
where id = …

```

---

<div class="post-metadata">

**Author:** ![KenWhitesell](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.djangoproject.com/kenwhitesell/32/280_2.png) [@KenWhitesell](https://forum.djangoproject.com/u/KenWhitesell)\
**Post date:** [April 3, 2021, 1:18am UTC](https://forum.djangoproject.com/t/help-with-many-to-many-formsets/7328/10 "2021-04-03T01:18:55Z")

</div>

Actually, I just got done posting these notes on this on another thread -  
Briefly:

- Remove directorylink from the list of fields - you don’t want Django generating that form field
- Define a field for the description in the form.
- Override the ` __init__ ` method to set the field if the form is bound to an object
- Override the `save` method to save that field in the related object (if you’re updating it)

Keep in mind that a ModelForm is a form, with the additional functionality of Django creating form fields for you in a more “declarative” mode. So anything you could do in a form, you can also do in a ModelForm. A ModelForm doesn’t remove any functionality from a Form.

---

<div class="post-metadata">

**Author:** ![typonaut](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.djangoproject.com/typonaut/32/2316_2.png) [@typonaut](https://forum.djangoproject.com/u/typonaut)\
**Post date:** [April 3, 2021, 1:25pm UTC](https://forum.djangoproject.com/t/help-with-many-to-many-formsets/7328/11 "2021-04-03T13:25:12Z")

</div>

Ok, I think I am pretty close. The `description` field appears on the `formset`, but is empty. I print the content I expect to see in the `description` in the terminal, and that looks exactly as I expect – so a bit confused as to how I get these values on the browser page.

```auto
class DirectoryChoices(forms.ModelForm):
	model = DirectoryLink_profiles
	
	def __init__ (self, *args, **kwargs):
		
		super(). __init__ (*args, **kwargs)
	
		if 'fields' in dir(self) :
			self.base_fields['description'] = forms.CharField()
			self.fields['description'] = forms.CharField()
			self.fields['description'].initial = 'this is hard work'
			print('fields: ', (self.fields['description'].initial))
		
			self.fields['description'].initial = kwargs['instance'].directorylink.description
			########################################
			print('self.fields[description]: ', (self.fields['description'].initial))
			########################################

		print('kwargs yourself dir: ', dir(kwargs))
		print('kwargs yourself dict: ', dict(kwargs))
		#initial = kwargs.get('initial', {})
		#self.fields['description'] = forms.CharField()
		self.fields['description'] = 'hell, initial description'
		#description = 'hell, initial description'
		
		super(DirectoryChoices, self). __init__ (*args, **kwargs)
	
	class Meta:
		model = DirectoryLink_profiles
		
		fields = (
			'uselesstext',
			'profile',
			)
			
		widgets = {
            'directorylink': forms.TextInput(attrs={'placeholder': 'Do not edit!'}),
        }

```

If I put `description` in the `fields` list I get an error that the field is unknown.

---

<div class="post-metadata">

**Author:** ![KenWhitesell](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.djangoproject.com/kenwhitesell/32/280_2.png) [@KenWhitesell](https://forum.djangoproject.com/u/KenWhitesell)\
**Post date:** [April 3, 2021, 4:24pm UTC](https://forum.djangoproject.com/t/help-with-many-to-many-formsets/7328/12 "2021-04-03T16:24:30Z")

</div>

You are really close - you don’t want to change the “initial” attribute on the field, but you do want to change the initial dict on the form, with the key of the field name.  
e.g. If your form field is `description`, then after calling `super(). __init__ (...)`, you can the initial value using:  
`self.initial['description'] = "some appropriate value"`

---

<div class="post-metadata">

**Author:** ![typonaut](https://sea2.discourse-cdn.com/flex026/user_avatar/forum.djangoproject.com/typonaut/32/2316_2.png) [@typonaut](https://forum.djangoproject.com/u/typonaut)\
**Post date:** [April 3, 2021, 5:37pm UTC](https://forum.djangoproject.com/t/help-with-many-to-many-formsets/7328/13 "2021-04-03T17:37:26Z")

</div>

Actually seems you can do it both ways:

```auto
self.fields['description'].initial = kwargs['instance'].directorylink.description

```

and

```auto
self.initial['description'] = kwargs['instance'].directorylink.description

```

That was not the cause of empty `description` fields, rather, this was, at the bottom of ` ___init__ `:

```auto
super(DirectoryChoices, self). __init__ (*args, **kwargs)

```

Take that out and the fields are populated.

Thanks for the feedback.