Django modelformset_factory custom validation with clean

Using Django Forms & APIs
1

Hello,

I’m trying to make a custom validation on a model formset,

to do so I’ve defined a class as follow:


class BaseUnloadFormset(BaseModelFormSet):
    def clean(self):
        super().clean()

        for form in self.forms:
            cd = form.cleaned_data
            whouse = cd['whouse']
            company = cd['company']
            product = cd['product']
            quantity = cd.get('quantity', None)
            movement_date = cd['movement_date']
            if quantity:
                helper = StockHelper(product, whouse, company, movement_date)
                if quantity > helper.stock:
                    raise ValidationError(f'Attenzione quantità massima disponibile di {product}: {helper.stock}')

and then I use it in the modelformset_factory as so:

UnloadFormSet = modelformset_factory(
    model=Unload,
    fields='__all__',
    extra=5,
    form=UnloadForm,
    formset=BaseUnloadFormset
)

Problem by using the base modelformset, I will get a key error on every form, even those that I left blank, and should not validate. Moreover, I’ll get a key error even on the first form if I just fill the first field. Instead of getting the “this field is required” built-in red validation message, the server will go down with a key error.

this is the formset with just the first form filled:

formset-with-data
formset-with-data3072×1728 384 KB

the error I get is:
KeyError at /Warehouse/unload/

‘whouse’


class BaseUnloadFormset(BaseModelFormSet):
    def clean(self):
        super().clean()

        for form in self.forms:
            cd = form.cleaned_data
            whouse = cd['whouse'] # <-- It crashes here, but on the second form not the first

Basically it crashes on the second form, where I’ve not selected the warehouse…which it should not because being a formset, if the form is not filled it should be ignored… or else I should get the django red error.

If I remove the formset=BaseUnloadFormset everything works correctly, but I loose the custom validation that I need.

I tried to follow the docs https://docs.djangoproject.com/en/3.2/topics/forms/modelforms/#overriding-clean-on-a-modelformset, but apparently I’m doing something wrong…

Any suggestions on the matter?

thank you very much

2

What exactly are you trying to do in your clean function?

There are potentially a couple things wrong here, but I can’t tell without knowing what your objective is.

3

Thank you for the reply Ken,

the app is called Warehouse, nothing too fancy.
The Unload class takes care of recording in the DB all of the unloads. In the formset you have to specify the warehosue from where you are unloading, the company ( this app is made for a group, not open to the public ), the product, the quantity, and the date when the product was moved.

I then used a custom class I made (StockHelper) to make sure you can not remove more product than the warehouse has available, if that is the case I return a validation error informing of the product and the max quantity available.

Hope this explains my purpose.

thank you very much

4

So how would you write a query to retrieve the number of product that the warehouse has available?

5

This is the StockHelper Class:


class StockHelper():

    def __init__(self, product, whouse, company, date):
        self.product = product
        self.whouse = whouse
        self.company = company
        self.date = date
        self.message = 'Attenzione quantità errate, controllare!'
        self.is_available = False

        loads = Load.objects.filter(product=self.product, whouse=self.whouse, company=self.company, movement_date__lte=self.date)

        tot_loads = loads.aggregate(Sum('quantity'))
        tot_unloads = Unload.objects.filter(product=self.product, whouse=self.whouse, company=self.company,movement_date__lte=self.date).aggregate(Sum('quantity'))
        tot_transfer_from = Transfer.objects.filter(product=self.product, from_whouse=self.whouse, company=self.company, movement_date__lte=self.date).aggregate(Sum('quantity'))
        tot_transfer_to = Transfer.objects.filter(product=self.product, to_whouse=self.whouse, company=self.company, movement_date__lte=self.date).aggregate(Sum('quantity'))
        tot_return_from_cs = LoadFromConstructionSite.objects.filter(product=self.product, whouse=self.whouse, company=self.company, movement_date__lte=self.date).aggregate(Sum('quantity'))

        self.stock = 0
        if tot_loads['quantity__sum']:
            self.stock += tot_loads['quantity__sum']
        if tot_transfer_from['quantity__sum']:
            self.stock -= tot_transfer_from['quantity__sum']
        if tot_unloads['quantity__sum']:
            self.stock -= tot_unloads['quantity__sum']
        if tot_transfer_to['quantity__sum']:
            self.stock += tot_transfer_to['quantity__sum']
        if tot_return_from_cs['quantity__sum']:
            self.stock += tot_return_from_cs['quantity__sum']

        if self.stock >= 0:
            self.message = 'OK'
            self.message_class = 'text-success'
        else:
            self.message = 'Attenzione quantità Negativa'
            self.message_class = 'text-danger'

        if self.stock > 0:
            self.is_available = True

        return

First I get the loads filtering for product, warehouse, company, movement_date__lte.
then I Calculate the total quantity for loads,

then I do the same for unloads, transfers from and to ( products could be moved from one warehouse to another one ), and returns ( product could return from the construction site )

At this point I start the counter self.stock = 0, and I add and remove depending if the product is going in or out of the warehouse.

At the end, I should be left with the actual stock of a certain product, in a warehouse, of that company up to the date selected.

Hope it is clear

thank you

7

Ok, StockHelper isn’t a model. It’s just a function that has been made into a class. (I’m not sure how that helps anything here, but ok…)

I’d verify what “helper” is and what its values are after creating the instance of StockHelper. Easiest way would be to add a couple of print calls after the helper= statement.

8

I’ve updated the original post, perhaps I was not clear. I never get an error in the StockHelper class.
It crashes before, looking for the cleaned data.
I’ve shown in the picture one case. The first form is filled correctly…on the second form the first field is the warehouse (cd[‘whouse’]), which can’t be find in the cleaned_data.

I find this behavior strange, as django should ignore the second form since it is not filled anywhere…

if for example, in the second form I fill the first field ( whouse ), then the server will crash on the next one, instead of displaying django built-in validation as follow:

normal-error
normal-error3072×1728 392 KB

I was able to make this screenshot by removing the formset in the modelformset_factory as follow:

UnloadFormSet = modelformset_factory(
    model=Unload,
    fields='__all__',
    extra=5,
    form=UnloadForm,
    # formset=BaseUnloadFormset
)

Which means that without “formset=BaseUnlaodFormset”, the formset behave correctly…which is strange beacause I’m not doing anything fancy…

hope this makes it more clear.

Thank you very much

9

Ok, I’m following you now.

What does your view look like for this? It appears as if you may be building the formset in the post differently than how it was built for the get in terms of initial data.

10

this is the view:


@login_required
def unload(request):
    helper = UnloadFormSetHelper()
    context = {
        'helper': helper,
        'title': 'Nuovo SCarico',
    }
    if request.method == 'GET':
        formset = UnloadFormSet(queryset=Unload.objects.none())
        load_user_companies_in_formset(request, formset)
        context['formset'] = formset

    elif request.method == 'POST':
        formset = UnloadFormSet(request.POST)
        context['formset'] = formset
        load_user_companies_in_formset(request, formset)
        if formset.is_valid():
            formset.save()
            messages.success(request, 'Scarico salvato correttamente', fail_silently=True)
            return HttpResponseRedirect(reverse('warehouse:dashboard'))
        else:
            return render(request, 'warehouse/unload.html', context)

    return render(request, 'warehouse/unload.html', context)

the load_user_companies_in_formset() function just does this:

def load_user_companies_in_formset(request, formset):
    for form in formset:
        form.fields['company'].queryset = request.user.profile.companies

It load all the companies for which the user is enabled.

Thank you

11

My first off-the-cuff reaction is that:

You’re creating the formset with no Unload objects as part of the formset, but you’re reconstituting it on the post without the queryset:

I’ll look a little deeper, but the first thing I’d try would be to add the queryset here.

You generally want to make sure that the formset being rebuilt on the post is built from the same data as was built on the get. That way, the forms can be compared to the initial data to see if anything was changed.

(This has generally been the cause for me whenever I’ve encountered unfilled forms as throwing validation errors.)

12

I tried adding the same queryset on POST, but it does not change anything.
I’ve been having a hard time finding some example online of modelformset clean validation ( apart from the docs ). It looks pretty straight forward, but nonetheless just adding the validation completely screws the formset behaviour, where previously I uset on normal formsets and everything worked fine.

Thank you anyway Ken for your time and effort. It is very appreciated.

13

@KenWhitesell
Found a solution that makes everything work again!

basically you have to test if there are any errors ( like a normal formset), then check if the cleaned_data are there before executing the custom validation:

class BaseUnloadFormset(BaseModelFormSet):
    def clean(self):
        super().clean()

        if any(self.errors): # <<-- like normal formset validation, makes internal django validation work again!
            return
        
        for form in self.forms:
            cd = form.cleaned_data
            if cd: # <<<----- if there are no cleaned_data the form should be skipped

                # get form position for better message
                form_position = self.forms.index(form) + 1
                form_position_message = f' nel form n. {form_position}!'

                whouse = cd['whouse']
                company = cd['company']
                product = cd['product']
                quantity = cd.get('quantity', None)
                movement_date = cd['movement_date']
                if quantity:
                    helper = StockHelper(product, whouse, company, movement_date)
                    if quantity > helper.stock:
                        raise ValidationError(f'Attenzione {form_position_message} Quantità massima disponibile di {product}: {helper.stock}')

what was added are this two:

if any(self.errors): # <<-- like normal formset validation, makes internal django validation work again!
            return
...

            if cd: # <<<----- if there are no cleaned_data the form should be skipped

I’m glad I could make it work…hope it can helpful to others.
Could it be that the docs are not fully explanatory?

14

Thanks for the follow-up!

Ok, errors is a property attribute. If it hasn’t been populated yet, it results in calling full_clean. I think I’d want to read this a little more closely before making further comments.

15

Sure thank you very much.
Let me know if you need more details.