altering queryset in a field of generic CreateView form

I have a generic form generated by a CreateView. The model is an Ingredient that consists of an Element and a quantity. Database constraint allows only unique Elements to be added to the recipe.

So the queryset for the element field needs to be Elements that are not already in the recipe. I understand how to make that queryset, but can only find a convoluted method to pass this into the template. This is what I am doing now in get_context_data()

def get_context_data(self, **kwargs):
    context = super().get_context_data(**kwargs)

    # get the right queryset.  This part is fine.
    qe = Element.objects.all()       # list of all elements
    qr = recipe.ingredients.all()    # gives list of elements already in this recipe
    qs =  qe.differenece(qr).order_by('name')

    # this next works but feels very awkward.  Writing into one of the form's field's queryset value.
    f = context['form']
    fe = f.fields['element']
    fe.queryset = qs
    return context

My question is: Can I pass this queryset into super().get_context_data() as a keyword argument? If so, how?

Thank you in advance.

To directly answer your question, CreateView uses FormMixin as one of its components. FormMixin sets kwargs[‘form’] to the form to be rendered. If you need to access the form here, that’s how you can get to it and modify it.

<opinion>
Personally, I wouldn’t do it this way. The problem with this approach is that it handles the creation of the form for the GET operation but does nothing on the POST.
I’d create a form for that model with the queryset defined to limit the choices available. That’s going to allow the form to validate the selection when the form has been submitted, preventing someone from injecting an invalid value into the recipe. (Or preventing two people from submitting with the same ingredient at effectively the same time.)
<opinion>

1 Like

Thank you, Ken. I am still new to Django and want to do things the most direct way, which led me to post this. I respect your opinion and will do that.

The only way I could determine how to do this using a class based CreateView is to:

  1. create a model form
  2. override the get_form() method in the view:
    def get_form(self, *args, **kwargs):
        logging.debug('RecipeAddIngredient')
        form = super(RecipeAddIngredient, self).get_form(*args, **kwargs)
        if (form.is_bound == False):
            # so in the GET.   Limit choices queryset.
            glaze = Recipe.objects.get(id = self.kwargs['pk'])
            qse = Element.objects.all()
            qsg = recipe.ingredients.all()
            g_qs = qse.difference(qsg).order_by('name')
            form.fields['element'].queryset = g_qs
        return form

Is this what you meant? I believe the POST does the validate just fine. If this is not what you meant, how else?

Unfortunately CreateView does not call get_queryset(). That would have been simple.

Nope, get_queryset is a method on a view, not for a data element within a form. (get_queryset is to retrieve the entities being viewed in something like a ListView - a different requirement than the filtering of choices in a form.)

You would create your form with this field based on a ModelChoiceField, using a custom queryset for limiting the options.

I’m getting the impression that you might be confusing a view with a form. A view is a representation of an entire page being rendered, a form is just one component on that page. As such, a form is a different object, usually defined in a forms.py file. (In your case, I believe you’re looking to use a ModelForm.) The get_form method then just creates an instance of that form. Within the model form definition, you have the ability to override the definitions of specific fields.

Ken,

I don’t see how the form will get the recipe_id (needed to make the limiting queryset). The view can access recipe_id through kwargs[‘pk’]. How can the form retrieve that information?

If you follow the chain of functions through CreateView, you’ll see that get_form creates an instance of the form passing a keyword dictionary returned from the function get_form_kwargs. This becomes a hook you can use to pass keyword-named args to the form initializer. That would give you something like this: (I’m winging it - there’s probably an error or three here)

def get_form_kwargs(self):
    kwargs = super().get_form_kwargs()
    kwargs['recipe'] = self.kwargs['pk'] 
    # Note there's a difference between the local variable kwargs used by
    # the function and the instance variable self.kwargs
    return kwargs

Then, your model form:

class MyModelForm(ModelForm):
    def __init__(self, *args, **kwargs):
        self.recipe = kwargs.pop('recipe', None)
        super().__init__(*args, **kwargs)

    class Meta:
        model = MyModel
        fields = [...]

… and then you have an instance variable named recipe that is available in your form.

Thank you again Ken. I did what you said and it works fine. The way I limited the queryset is by setting the queryset attribute in the [‘element’] field of the unbound form.

    def __init__(self, *args, **kwargs):
        recipeid = kwargs.pop('recipe', None)
        super().__init__(*args, **kwargs)
        if not self.is_bound:
            self.recipe = Recipe.objects.get(id = recipeid)
            qsg = self.recipe.ingredients.all()
            qse = Element.objects.all()
            qs = qse.difference(qsg).order_by('name')
            self.fields['element'].queryset = qs

I thought that since recipeid is now available to the form I thought to use it in the Meta subclass doing something like:

    class Meta:
        model = Ingredient
        fields = ['element', 'quantity']
        querysets = { 'element' : limiting_queryset }

but since limiting_queryset is dynamic as above in the init_ method, I could not figure out how to make that happen. I believe the current scheme has the exact same effect even though it feels a bit clumsier putting the calculation in the init method.

While I’m still looking at your options for the Meta definition for this, I would say that you might not want to limit this filtering to the unbound form.

Assuming this is an absolute restriction on data entry, you’d want this level of validation performed both on the construction and the submittal of the form, to prevent either a race condition allowing for unintended element duplication or malicious actor intentionally submitting invalid data creating duplication element references. (never implicitly trust input supplied from the browser.)

Ken

I originally did not have the restriction of only doing this on an unbound form. However, without that I get this error:

MultipleObjectsReturned at /glazeman/glaze/4/addc
get() returned more than one Element -- it returned more than 20!
Request Method:	POST
Request URL:	http://localhost:8000/glazeman/glaze/4/addc
Django Version:	3.1.1
Exception Type:	MultipleObjectsReturned
Exception Value:	
get() returned more than one Element -- it returned more than 20!

So I figured that altering the queryset of the bound form was the culprit. Restricting it to a bound form made the error go away and everything appears to work as intended. Hadn’t considered a race condition yet.

That’s, uhhh, interesting. The only get I see from what you’ve posted is this:

and assuming that the pk of Recipe is id, then that retrieving multiple values seems really wrong to me. Can you verify that the error is being thrown on the statement above?
(Also, you don’t by any chance have a form field named recipe that your specific usage of self.recipe is stomping on, do you? I’m not sure there’s even a need to assign that recipe instance to an attribute in the form. You probably could get away with just using it as a method-local variable)

Stepping through with Pdb, the error comes from the form_valid() call.

Hmmm… that’s a bit of a head-scratcher.

Since you’ve got something working, I’m guessing it’s reaching the point where it’s probably more effective for you to just set this aside and keep moving forward. For me to be able to provide any further assistance, I’d likely end up needing to see the entire view and form here, if only to verify that there wasn’t something elsewhere causing this issue.
It’s your call - I’m more than happy to devote some odd minutes here and there to resolve this (I love a good puzzle), but I can understand if you just want to let it go and move on.

Ken

Ken,

Thank you for taking the time to check on this error. I changed some field names to verify there is no overlap and reran it twice. First time not setting a limit queryset when form.is_bound and getting no error. Second time through setting limit queryset on both the bound and unbound cases. Error only happens on the second case.

Below are the model, form and view as well as console output from both the good and error runs. Note debug logging in console output and view and form code.

==========================================================
models.py

class Ingredient(models.Model):
    element = models.ForeignKey(Element, on_delete=models.CASCADE)
    glaze = models.ForeignKey(Glaze, on_delete=models.CASCADE)
    quantity = models.DecimalField(max_digits=6, decimal_places=3, null = True, blank = True)         # % of element in glaze mixture
    
    def __str__(self):
        s = f'{self.glaze.name} : {self.element.name} | above the line'
        return s
    
    
    # only one element/glaze pairing for colorant or element 
    class Meta:
        constraints = [
            models.UniqueConstraint(fields = ['element_id', 'glaze_id'], name = 'duplicate_element_in_glaze')
        ]

=================================================================
forms.py

class AddGlazeIngredientForm(ModelForm):
    def __init__(self, *args, **kwargs):
        logging.debug('AddGlazeIngredientForm')
        gid = kwargs.pop('gpk', None)               # this is pk of glaze object
        super().__init__(*args, **kwargs)
        # if not self.is_bound:
        # build queryset that contains no objects already in glaze recipe
        g = Glaze.objects.get(id = gid)             # current glaze object whose pk is passed through view kwargs
        qsg = g.ingredients.all()                   # all elements in glaze
        qse = Element.objects.all()                 # all allowable elements
        qs = qse.difference(qsg).order_by('name')   # queryset difference
        self.fields['element'].queryset = qs
        self.which = 'ingredient'                   # used in common template
        # end block if not self.is_bound
        logging.debug('leaving AddGlazeIngredientForm')
      
    class Meta:
        model = Ingredient
        fields = ['element', 'quantity']

================================================================
views.py

class GlazeAddIngredient(SuccessMessageMixin, LoginRequiredMixin, generic.CreateView):
    form_class = AddGlazeIngredientForm
    template_name = 'glazes/add_to_glaze_form.html'
    success_url = reverse_lazy('glazes:glazes list')
    success_message = 'New ingredient added.'
    
    def get_success_url(self):
        return reverse_lazy('glazes:glaze detail', args=[self.kwargs['pk']])   # back to Glaze update form

    def form_valid(self, form):
        logging.debug('GlazeAddIngredient')
        form.instance.glaze = Glaze.objects.get(id = self.kwargs['pk'])
        return super(GlazeAddIngredient, self).form_valid(form)

    def get_form_kwargs(self, **kwargs):
        # Note there's a difference between the local variable kwargs used by
        # this function and the instance variable self.kwargs
        logging.debug('GlazeAddIngredient')
        kwargs = super().get_form_kwargs()
        kwargs['gpk'] = self.kwargs['pk']
        return kwargs

================================================================
console log with the code block under
if not form.is_bound:

Django version 3.1.1, using settings 'glazeman.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CTRL-BREAK.
[20/Oct/2020 11:00:13] "GET /glazeman/glaze/4 HTTP/1.1" 200 7882
[20/Oct/2020 11:00:14] "GET /static/glazes/icons/site.webmanifest HTTP/1.1" 304 0
DEBUG: 11:00:16 views get_form_kwargs 201 GlazeAddIngredient
DEBUG: 11:00:16 forms __init__ 28 AddGlazeIngredientForm
DEBUG: 11:00:16 forms __init__ 38 leaving AddGlazeIngredientForm
[20/Oct/2020 11:00:16] "GET /glazeman/glaze/4/add HTTP/1.1" 200 5523
DEBUG: 11:00:29 views get_form_kwargs 201 GlazeAddIngredient
DEBUG: 11:00:29 forms __init__ 28 AddGlazeIngredientForm
DEBUG: 11:00:29 forms __init__ 38 leaving AddGlazeIngredientForm
DEBUG: 11:00:29 views form_valid 194 GlazeAddIngredient
[20/Oct/2020 11:00:29] "POST /glazeman/glaze/4/add HTTP/1.1" 302 0
[20/Oct/2020 11:00:29] "GET /glazeman/glaze/4 HTTP/1.1" 200 8343
C:\Users\Ken\Python\Django\glazeman\glazes\forms.py changed, reloading.
INFO: 11:01:41 autoreload trigger_reload 236 C:\Users\Ken\Python\Django\glazeman\glazes\forms.py changed, reloading.
DEBUG: 11:01:41 proactor_events __init__ 623 Using proactor: IocpProactor
Watching for file changes with StatReloader
Performing system checks...

================================================================
console log without check on form.is_bound

Django version 3.1.1, using settings 'glazeman.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CTRL-BREAK.
DEBUG: 11:02:12 views get_form_kwargs 201 GlazeAddIngredient
DEBUG: 11:02:12 forms __init__ 28 AddGlazeIngredientForm
DEBUG: 11:02:13 forms __init__ 38 leaving AddGlazeIngredientForm
[20/Oct/2020 11:02:13] "GET /glazeman/glaze/4/add HTTP/1.1" 200 5475
DEBUG: 11:02:18 views get_form_kwargs 201 GlazeAddIngredient
DEBUG: 11:02:18 forms __init__ 28 AddGlazeIngredientForm
DEBUG: 11:02:18 forms __init__ 38 leaving AddGlazeIngredientForm
Internal Server Error: /glazeman/glaze/4/add
Traceback (most recent call last):
  File "C:\Users\Ken\Python\Django\glazeman\venv\lib\site-packages\django\core\handlers\exception.py", line 47, in inner
    response = get_response(request)
  File "C:\Users\Ken\Python\Django\glazeman\venv\lib\site-packages\django\core\handlers\base.py", line 179, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "C:\Users\Ken\Python\Django\glazeman\venv\lib\site-packages\django\views\generic\base.py", line 70, in view
    return self.dispatch(request, *args, **kwargs)
  File "C:\Users\Ken\Python\Django\glazeman\venv\lib\site-packages\django\contrib\auth\mixins.py", line 52, in dispatch
    return super().dispatch(request, *args, **kwargs)
  File "C:\Users\Ken\Python\Django\glazeman\venv\lib\site-packages\django\views\generic\base.py", line 98, in dispatch
    return handler(request, *args, **kwargs)
  File "C:\Users\Ken\Python\Django\glazeman\venv\lib\site-packages\django\views\generic\edit.py", line 172, in post
    return super().post(request, *args, **kwargs)
  File "C:\Users\Ken\Python\Django\glazeman\venv\lib\site-packages\django\views\generic\edit.py", line 141, in post
    if form.is_valid():
  File "C:\Users\Ken\Python\Django\glazeman\venv\lib\site-packages\django\forms\forms.py", line 177, in is_valid
    return self.is_bound and not self.errors
  File "C:\Users\Ken\Python\Django\glazeman\venv\lib\site-packages\django\forms\forms.py", line 172, in errors
    self.full_clean()
  File "C:\Users\Ken\Python\Django\glazeman\venv\lib\site-packages\django\forms\forms.py", line 374, in full_clean
    self._clean_fields()
  File "C:\Users\Ken\Python\Django\glazeman\venv\lib\site-packages\django\forms\forms.py", line 392, in _clean_fields
    value = field.clean(value)
  File "C:\Users\Ken\Python\Django\glazeman\venv\lib\site-packages\django\forms\fields.py", line 149, in clean
    value = self.to_python(value)
  File "C:\Users\Ken\Python\Django\glazeman\venv\lib\site-packages\django\forms\models.py", line 1274, in to_python
    value = self.queryset.get(**{key: value})
  File "C:\Users\Ken\Python\Django\glazeman\venv\lib\site-packages\django\db\models\query.py", line 433, in get
    raise self.model.MultipleObjectsReturned(
glazes.models.Element.MultipleObjectsReturned: get() returned more than one Element -- it returned more than 20!
ERROR: 11:02:18 log log_response 224 Internal Server Error: /glazeman/glaze/4/add
Traceback (most recent call last):
  File "C:\Users\Ken\Python\Django\glazeman\venv\lib\site-packages\django\core\handlers\exception.py", line 47, in inner
    response = get_response(request)
  File "C:\Users\Ken\Python\Django\glazeman\venv\lib\site-packages\django\core\handlers\base.py", line 179, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "C:\Users\Ken\Python\Django\glazeman\venv\lib\site-packages\django\views\generic\base.py", line 70, in view
    return self.dispatch(request, *args, **kwargs)
  File "C:\Users\Ken\Python\Django\glazeman\venv\lib\site-packages\django\contrib\auth\mixins.py", line 52, in dispatch
    return super().dispatch(request, *args, **kwargs)
  File "C:\Users\Ken\Python\Django\glazeman\venv\lib\site-packages\django\views\generic\base.py", line 98, in dispatch
    return handler(request, *args, **kwargs)
  File "C:\Users\Ken\Python\Django\glazeman\venv\lib\site-packages\django\views\generic\edit.py", line 172, in post
    return super().post(request, *args, **kwargs)
  File "C:\Users\Ken\Python\Django\glazeman\venv\lib\site-packages\django\views\generic\edit.py", line 141, in post
    if form.is_valid():
  File "C:\Users\Ken\Python\Django\glazeman\venv\lib\site-packages\django\forms\forms.py", line 177, in is_valid
    return self.is_bound and not self.errors
  File "C:\Users\Ken\Python\Django\glazeman\venv\lib\site-packages\django\forms\forms.py", line 172, in errors
    self.full_clean()
  File "C:\Users\Ken\Python\Django\glazeman\venv\lib\site-packages\django\forms\forms.py", line 374, in full_clean
    self._clean_fields()
  File "C:\Users\Ken\Python\Django\glazeman\venv\lib\site-packages\django\forms\forms.py", line 392, in _clean_fields
    value = field.clean(value)
  File "C:\Users\Ken\Python\Django\glazeman\venv\lib\site-packages\django\forms\fields.py", line 149, in clean
    value = self.to_python(value)
  File "C:\Users\Ken\Python\Django\glazeman\venv\lib\site-packages\django\forms\models.py", line 1274, in to_python
    value = self.queryset.get(**{key: value})
  File "C:\Users\Ken\Python\Django\glazeman\venv\lib\site-packages\django\db\models\query.py", line 433, in get
    raise self.model.MultipleObjectsReturned(
glazes.models.Element.MultipleObjectsReturned: get() returned more than one Element -- it returned more than 20!
[20/Oct/2020 11:02:18] "POST /glazeman/glaze/4/add HTTP/1.1" 500 114719

Just wanted to drop you a quick line that:

  • I haven’t forgotten about this
  • I don’t have an answer for you yet
  • I’m still digging, trying to understand exactly what’s going on.

Ken

Try applying .distinct() to the queyset on the element field.

self.fields['element'].queryset = qs.distinct()

I know it’s possible to have duplicate instances returned when filters cross multiple many to many fields. I’m not sure how the difference function is working. It might be easier to reason out if you view the query via the Django Debug Toolbar or print(qs.query).

Another way to write it would be:

self.fields['element'].queryset = Element.objects.exclude(ingredient__glaze=gid).distinct()
1 Like

Tim,

That was the answer! Removing .order_by(‘name’) was the key. Thank you for suggesting this, as well as the proper way of using the .exclude method.

I replaced:

    qsg = self.g.ingredients.all()
    qse = Element.objects.all()
    qs = qse.difference(qsg).order_by('name')

with

    qs = Element.objects.exclude(ingredient__glaze=gid)

The queryset returned is ordered by name, almost certainly because the Element model includes:

    class Meta:
        ordering = ['name']

Thank you both for spending time and effort to help with this.

1 Like