Handling infinitely nested product configuration forms?

Using Django Forms & APIs
1

I’m building an order system and facing the following challenge.

An Order can contain multiple products (represented as OrderItem). Some of these products can have subproducts that are part of the order but specifically relate to a particular main product.

A simple example would be a car order: You can order multiple cars (main products), and for one of these cars, you might want to add a set of premium tires (a subproduct) that belongs specifically to that car.

This is where the item field in OrderItem comes into play—if you order two cars and two different sets of tires, the system needs to track which set of tires belongs to which car.

Where it gets interesting is that each subproduct can, in turn, have its own subproducts. For example, the premium tires might have custom tire caps as an additional configuration option. This nesting continues indefinitely until the configuration rules are exhausted.

class Order(models.Model):
customer = models.ForeignKey(Customer, on_delete=models.CASCADE)

class OrderItem(models.Model):
item = models.ForeignKey("self", on_delete=models.CASCADE, null=True, blank=True)
order = models.ForeignKey(Order, on_delete=models.CASCADE, related_name="items")
product = models.ForeignKey(Product, on_delete=models.CASCADE)

class Product(models.Model):
name = models.CharField(max_length=255)

I’ve implemented this using Django and HTMX. Each time a product is selected, I fetch its related products and render the corresponding form. If one of these subproducts is selected, I fetch and render its related products, and so on.

The key part is that for each subproduct form, I assign it a prefix based on its parent product. This results in form prefixes like configuration-1-5-2-17 where:

1 is the main product

5 is a subproduct

2 is a sub-subproduct

17 is a sub-sub-subproduct

Now, my main concern is whether this is the best approach . Initially, I tried using FormSets, but I realized that I’m dealing with multiple different forms rather than the same form repeated multiple times.

Also, I am solely concerned about the backend here (how to build the forms, leverage Django’s tools, etc.), but too much about the frontend.

2

Welcome @0584265c !

Does it work? That should be the primary criteria. Any solution that works is infinitely superior to all solutions that don’t.

We do something like what you’re doing, only we do use nested formsets.

See my description at Multiples of one formset - #12 by KenWhitesell

While our current implementation is limited to 3 levels, that’s a business requirements and not a technical requirement. It could be expanded deeper than that.

3

Hi @KenWhitesell,

I was hoping you would reply—I saw your post, and that’s actually where I got the idea from!

Regarding the question of whether it works, I’ve primarily created the CreateView, and it seems to work. However, with this approach, I have to process the POST request manually to fetch all the data. See:

# forms.py
class ProductSelectForm(forms.Form):
    product = forms.ModelChoiceField(queryset=None, label="Select Product", required=False)

    def __init__(self, *args, queryset=None, **kwargs):
        super().__init__(*args, **kwargs)
        self.fields["product"].queryset = queryset or Product.objects.none()

# views.py
class OrderCreate(View):
    def post(self, request):
        form = ProductSelectForm(request.POST, prefix="main")

        if form.is_valid():
            order = Order.objects.create(user=request.user)
            self.process_config(request.POST, order, "main")
            ...

    def process_config(self, post_data, order, prefix):
        parent_item = None

        while True:
            if f"{prefix}-product" not in post_data:
                break

            product_id = post_data.get(f"{prefix}-product")
            if not product_id:
                break

            config_product = Product.objects.get(id=product_id)
            config_item = OrderItem.objects.create(product=config_product, item=parent_item, order=order)

            parent_item = config_item
            prefix = f"{prefix}_{product_id}"

Given this context, my concern is that this approach might either be underutilizing Django’s capabilities (that I might not be aware of) or that this is actually the best way to handle it.

Do you happen to have any thoughts on this? Is there an alternative approach that leverages Django’s toolbox more effectively, or would that only add unnecessary complexity?

Thanks in advance!

4

If all form instances under a particular object are the same type of form, then there might be some benefit to implementing it as a formset.

Beyond that, if the form is defined with a prefix, then you can leverage the form by binding the form using the prefix.

In other words, if the subordinate form is a ProductSelectForm, then you can rebind it as ProductSelectForm(request.POST, prefix=f{prefix}-product).

Or, I might be misunderstanding what this portion of the code is trying to do - it would help my understanding if this were a more complete example.

5

Sure @KenWhitesell, I will expand it - Hopefully I can explain it clear:

We start with a Single ProductSelectForm for a main product (e.g. the different Cars).

class OrderCreate(View):
    def get(self, request):
        form = ProductSelectForm(queryset=Product.objects.filter(...), prefix="main")
        return render(request, "orders/form.html", {"form": form})

In our HTML, when selecting a specific car, it sends an HTMX request to check if the product has a any subproducts and if so, it renders a new form and appends it below in our HTML.

def hx_configuration(request):
    parent_form_prefix = request.GET.get("prefix", "")
    
    parent_product = Product.objects.get(...)
    sub_products_query = Product.objects.filter(...)

    form_prefix = f"{parent_form_prefix}_{parent_product.id}"
    sub_form = ProductSelectForm(prefix=form_prefix, queryset=sub_products_query)
    return render(request, ..., {"sub_form": sub_form})

So, in our HTML we might end up with an overview like this:

  • Car Form (prefix: main)
    • Tires Form (prefix: main_car_id)
      • Caps Form (prefix: main_car_id_tires_id)
    • Engine Form (prefix: main_car_id)
      • Oil Form (prefix: main_car_id_engine_id)
        • Filter Form (prefix: main_car_id_engine_id_oil_id)

In summary, the Tires Form is dynamically generated based on the Car selection, The Caps Form is dynamically generated based on the Tires selected and so on.

In the end all these forms (Car, Tires, Cap, Engine, etc.) are the same form (ProductSelectForm), yes - but they refer to different parent items. The Tires need to be linked to the Car, the Caps to the Tires, the Engine to the Car, the Oil to the Engine in our OrderItem.

Now, on POST I have all these dynamic forms that I need to somehow process. I have my initial form (Car) and from there I now that all subsequent forms have the prefix of the parent form (“main”) plus the selected product ID (main_car_id). From there it is a matter of
searching for all the forms using that method (as you saw in the process_config method).

I’m struggling with how to process these forms in a clean way. The current approach is a bit of a mess.

Again, thanks very much!

6

Short answer: No, I’m not aware of any “simple” solution to this.

Quoting from my previous post:

But what makes is relatively easy for us to process is that each parent element only has one subordinate element, which is a formset.

If you look at a sample prefix we use, ‘t-1-r-2-d-3’, the top level formset has the prefix ‘t’. When I bind the submitted data to the formset, I can then iterate over each form. Each form has its own index within the list of forms in the formset, the data associated with it, along with a subordinate formset. The prefix for the second level is ‘t-current_index-r’, which is intrinsically related to the top level instance that is currently being processed within the iteration of the parent formset.

This sample prefix would then be the fourth instance of the ‘d’ form within the third instance of the ‘r’ form within the second instance of the ‘t’ form.

This can be of arbitrary depth by making this processing a recursive function situation, where at any given point you’re processing the next level down, passing through the “prefix-to-this-point”, along with whatever other information may be needed such as the parent class type and pk.

7

Ok, ok - then I’ll start looking more into formsets. Where I’m getting lost is with “Each one of those forms is responsible for creating instances of a 2nd level form […] Finally, each one of those is responsible for creating a 3rd level formset”

Would it be possible to elaborate a bit more how the Formset creates its subordinate formset? Are you subclassing BaseInlineFormSet and with add_fields instantiate an inlineformset_factory with the prefixes you talked about? So, something like:

class OrderItemForm(forms.ModelForm):
    class Meta:
        model = OrderItem
        fields = ['product']

class BaseOrderItemFormSet(BaseInlineFormSet):
    def add_fields(self, form, index):
        super().add_fields(form, index)
        form.nested = SubItemFormSet(
            instance=form.instance,
            data=form.data if form.is_bound else None,
            files=form.files if form.is_bound else None,
            prefix=...
        )

SubItemFormSet = inlineformset_factory(
    OrderItem,
    OrderItem,
    form=OrderItemForm,
    formset=BaseOrderItemFormSet,
    fk_name='item', # -> parent item.
    fields=['product'],
    extra=1,
    can_delete=True
)
8

A Formset does not have subordinate formsets. A formset has a list of forms.

Each one of those forms may have a formset subordinate to it.

When binding a submitted form, the creation is done “manually”. There’s nothing special or customized about the process.

(That is one of the messier pieces of the code.)

In our case, since the structure is fixed at (now 4) levels, we know that every instance of the first three levels has a subordinate level to every instance of the form in the formset. That means that we always iterate over the forms on levels 1-3 to create a formset associated with that form, bound with post data.

One advantage of using formsets here is that you don’t need to have any explicit tests for the existance of a subordinate formset when you’re binding post data. If you create an instance of a formset with extra=0, and there’s no post data associated with the defined prefix, you get a formset with no forms.

This is why I’m thinking that a recursive process would likely work for you. Bind the post data for the top level, then iterate over the individual forms and create the subordinate formsets for each form instance.

As a general example to illustrate an idea, not intended to be complete or usable as is:

def create_formsets(prefix=''):
    current_formset = MyFormset(request.POST, prefix=prefix)
    for idx in range(len(current_formset.forms)):
        current_formset.forms[idx].subordinate = create_formsets(prefix=prefix+f'x-{idx}-')
    return current_formset

nested_structure = create_formsets()

Once the forms are bound to the submitted data, you can then process this nested structure. (Or, you could process it as needed within the function - it all depends upon what you need to do with the data.)