POST data yielding hidden form missing error

I am using a Formset to generate a set of text entries based on user input and then POST the data when the use selects submit. However, the data does not appear to POST, instead I get an error:

<ul class="errorlist nonfield"><li>(Hidden field TOTAL_FORMS) This field is required.</li><li>(Hidden field INITIAL_FORMS) This field is required.</li></ul>

  <div><input type="hidden" name="form-TOTAL_FORMS" id="id_form-TOTAL_FORMS"><input type="hidden" name="form-INITIAL_FORMS" id="id_form-INITIAL_FORMS"><input type="hidden" name="form-MIN_NUM_FORMS" id="id_form-MIN_NUM_FORMS"><input type="hidden" name="form-MAX_NUM_FORMS" id="id_form-MAX_NUM_FORMS"></div>

As far as I can tell I have no hidden fields defined in my models:

Models

class Dashboard(models.Model):
    dashboard = models.CharField(max_length=50)
    company = models.CharField(max_length=150, default="None")
   

    def __str__(self):
        return f"Company: {self.company} Dashboard: {self.dashboard}"

class Objectives(models.Model):
    dashboard=models.ForeignKey(Dashboard, on_delete=models.CASCADE)
    objective_row = models.IntegerField()
    objective_num = models.IntegerField()


class Objective_Colors(models.Model):
    user = models.ForeignKey(User, on_delete=models.CASCADE)
    objective = models.ForeignKey(Objectives, on_delete=models.CASCADE) 
    color=models.CharField(max_length=10)
    value=models.IntegerField()
    timestamp= models.TimeField(auto_now = True)


    def __str__(self):
        return f"{self.color} at {self.timestamp}"


class ObjectiveText(models.Model):
    user = models.ForeignKey(User, on_delete=models.CASCADE)
    objective = models.ForeignKey(Objectives, on_delete=models.CASCADE) 
    objective_text = models.CharField(max_length=1000, blank = True)
    timestamp= models.TimeField(auto_now = True)

Forms

ObjectiveTextFormSet = modelformset_factory(ObjectiveText, fields = ["objective_text", "objective"], 
                                            extra = 10, widgets={'objective_text': Textarea(attrs={'cols': 15, 'rows': 5})} )

class LoginForm(forms.Form):
    username = forms.CharField()
    password = forms.CharField(widget=forms.PasswordInput)

class LogoutForm(forms.Form):
    username = forms.CharField()

views

def home(request):
    
    # Create unique prefixes for each element
                
    formset_list = [ 
        ObjectiveTextFormSet
        (prefix=f'row-{i}') 
            for i in range(6)
            
    ]

    # Add the formset to context dictionary
    context = {
        'objectives': [1,2,3,4,5,6,7,8],
        'formset_list': formset_list,
    }

    if request.method == "POST":
        form = ObjectiveTextFormSet(request.POST)
        print(form)
        if  form.is_valid():
            print("Valid")
            objectives = form.save()
           # objectivetext = objectives_form.cleaned_data['objective_text']
           # objectivenum =  objectives_form.cleaned_data['objective_num']
        return render(request, 'ISO22301/home.html', context)
    else:
        return render(request, "ISO22301/home.html", context)

template

{% load static %}

<!DOCTYPE html>
<html lang = "en">

    <link rel="stylesheet" href="{% static 'css/styles.css' %}">

  <title>Project Management</title>


  <h1>Strategy Map</h1>
    <form method="POST" action="{% url "home" %}" enctype="multipart/form-data">
      {% csrf_token %}
      <table>
<head>
          <th>Project</th>
          {% for n in objectives %}
          <th>Objective</th>
          {% endfor %}
          <th>Outcome</th>
</head>
 <body>
            {{ formset.management_form }}
              {% for formset in formset_list %}
                      <tr>
                                {% for form in formset %}
                                <td>  {{ form.objective_text }} </td>
                                {% endfor %}
                      </tr>                        

              {% endfor %}
      
      </table> 
        <div>
          <input type="submit" value="Update Objectives">
        </div>
  </form>
  </body>
  </html>

DB (SQLLite)

That is true.

However, the formset defines a set of fields in the Management Form that are hidden.

You’re trying to render something named formset outside the loop in which it’s defined. (These two lines need to be reversed.)

Did that. Same error.

For the GET portion of your view, you’re not creating one instance of ObjectiveTextFormSet, you’re creating a list of 6 of them, each with a different prefix.

Your post data doesn’t contain submissions for one form set - it contains the data for 6 formsets.

When you’re binding the POST data to those individual forms, you need to bind that POST data to each of the forms in the list, specifying the same prefixes.

Ah. I thought it was one set of 60. I’ll dig into this and make changes. Thanks

When in doubt, check the rendered HTML and what is submitted in the post request.

Just to clarify, there are two unrelated things called “fields” here. There are “model fields” and “html form fields”. The error message is about the latter.

On an unrelated topic: why does the logout form have a username field? That seems wrong. The user is already known from request.user.

This is a common error with formsets: Django form using model_formset not submitting - Stack Overflow has a good answer that seems like a hint.

The stackoverflow answer could be solved by installing django-fastdev so that they would get errors on incorrect usages of template variables. It sounds like at least part of your problem would be helped in the same way. (As a side note, iommi’s EditTable will do this all much better)

If I understand what is in Using more than one formset in the docs “using-more-than-one-formset-in-a-view”, my formsets are prefixed with row-0 through row-6.

Looking at the HTML that appears to be the case:

<td>  <textarea name="row-0-0-objective_text" cols="15" rows="5" maxlength="1000" id="id_row-0-0-objective_text">

That appears to be the case when I look at the data variable returned:

row-0-0-objective_text

However, If I try to get the POST data I still do not get valid form data.

        formset =  ObjectiveTextFormSet(request.POST, prefix='row-0')
        if  formset.is_valid():

I tried just prefexing one row by getting rid of the for loop and using (prefix=‘row-0’) to ee if it was because I only got one row of data to check but that made no difference.

I’ve also tried ‘row-0-’ and ‘row-’

The POST data appears correct but I cannot seem to figure out how to get it from the POST request.

Could it be something to do with the name of the formset ObjectiveTextFormSet in the POST request as even when I remove the prefix and just render 1 row I get no valid data.

By no valid data I mean it doesn’t print “Valid” which is what I used to see if it is returning TRUE for .is_valid():

Got rid of the error and the POST Request has the hidden fields in it.

No particular reason; probably should just delete it

What are the errors being returned in the formset? (You might need to check all the individual forms to find all the errors.)

I think I have found the problem.

In my models, objective is a FK for Objective Text. It is used to relate objective text to the objective, which is the location on the table and sequential, i.e. the input on row objective 1 is Objective 0, down to row 6 objective 10 is objective 60.

Looking at the error message, the issue is POST is capturing the text input but has not included the related FK objective. I say that because I entered data in the first textarea,(row 0 Objective 1) and left the others blank; resulting in the ‘This field is required.’

The error is consistent across all 6 formsets, i.e. any textarea I enter txt generates an error.

I saw no other errors.

Error message:

[{'objective': ['This field is required.']}, {}, {}, {}, {}, {}, {}, {}, {}, {}]

The text I entered in the textarea appears in the POST area, based on the POST data shown when I cause an error such as a NameError by misspelling the formset name:

row-0-TOTAL_FORMS               '10'
row-0-INITIAL_FORMS             '0'
row-0-MIN_NUM_FORMS	            '0'
row-0-MAX_NUM_FORMS	            '1000'
row-0-0-objective_text	        'rewrwr'

I suspect I must include the objective id as part of each form but not sure how to include properly so I get a sequential list of objective ids that identify each textarea uniquely.

Any suggestions?

Notice that the field it’s complaining about is the objective field, not objective_text.

There are (at least) two options, depending upon whether these objectives being referenced exist or not at the time the page is being rendered.

If the Objectives exist for the ObjectiveText instances being used for that formset, then you would want to initialize the ObjectiveTextFormSet with the instances for that row, and render the objective field as a hidden field.

If the Objectives instance doesn’t exist, then you don’t want to include the objective field in the form at all. You’ll need to create it when the form is posted, then assign that instance to the field for each instance of the form in that formset before saving it.

Side note: Do not try to “manage” the IDs of the Objectives. Your application shouldn’t care what the IDs are of any model. Create or match them based upon the objective_row and objective_num fields.

Yea, as I looked at it I realized there was no way for the POST data to be tied to the objective.

How would I do that?

I was thinking along the same lines.

Right now, I have a set of objective_texts that are linked to the objectives, so I could use those instances to fix the problem; use the tag {{ form.field_objective.as_hidden }} before or after the field that is visible.

Then, when a new dashboard is created, create a set of objectives for that dashboard and a set of associated objective_text instances. That would only be done for a new dashboard.

[quote]Side note: Do not try to “manage” the IDs of the Objectives. Your application shouldn’t care what the IDs are of any model. Create or match them based upon the objective_row and objective_num fields.
[/quote]

That makes sense because there is no easy way to know what the id is, but joining dashboard - objective - objective_text lets SQL do its thing behind the scene.

That way, to create a new dashboard the process would be:
Create dashboard, then for each dashboard_name, create a set of objectives with a row/ num value, then similarly for objective_text create an entry for each objective using objective row/num to select which objective the text is for.

That would not require adding objective row / num to the objective_text model, correct? Just using it to identify the objective when creating the instance.

Does that make sense?

As a side note, how do I include the objective text in each objective textarea when I create the formset? Can I use a variable? The initial set would be blank, and as it gets populated I’d show the latest entry.

Yes, Absolutely.

This is the purpose of using a ModelFormSet. When creating the instance of the formset, you can pass a queryset parameter. This parameter associates a set of instances with the individual forms in the formset.

So, where you currently might have something like:
ObjectiveTextFormSet = modelformset_factory (ObjectiveText, ..., extra=10, ...)
this becomes:
ObjectiveTextFormSet = modelformset_factory (ObjectiveText, ..., extra=0, ...)
because you’re not creating new elements here - they already exist.

And, where you have:

Becomes something like:

formset_list = [ 
    ObjectiveTextFormSet(
         prefix=f'row-{i}',
         queryset=ObjectiveTextFormSet.objects.filter(
               objective__objective_row=i
             ).order_by('objective__objective_num')
        ) 
     for i in range(6)
]

Side note: If your objective_row numbers are “1-based” instead of “0-based”, you can change your iterator to be for i in range(1,7).

Then, to save these entries, after you bind the data in the post, it becomes something like:

for formset in formset_list:
    if formset.is_valid():
        formset.save()

Thanks. I’m gettng closer. After some research I had to change:

to

                    queryset=ObjectiveText.objects.filter(
                        objective__objective_row=i
                        ).order_by('objective__objective_num') 

since it appears you have to use models and not formsets in a queryset to get objects.

However, I still am apparently not getting post data.

I don’t get errors, but no data appears to be supplied as nothing is saved into the database ad adding a print(“Valid”) does not print.

If I print the formset I get a long list that repeats this:

  <div>
    <label for="id_row-6-9-objective">Objective:</label>



<select name="row-6-9-objective" id="id_row-6-9-objective">
  <option value="">---------</option>

  <option value="1">Objectives object (1)</option>

  <option value="2">Objectives object (2)</option>

  <option value="3">Objectives object (3)</option>

  <option value="4">Objectives object (4)</option>

  <option value="5">Objectives object (5)</option>

  <option value="6">Objectives object (6)</option>

  <option value="7">Objectives object (7)</option>

  <option value="8">Objectives object (8)</option>

  <option value="9">Objectives object (9)</option>

  <option value="10">Objectives object (10)</option>

  <option value="11">Objectives object (11)</option>

  <option value="12">Objectives object (12)</option>

  <option value="13">Objectives object (13)</option>

  <option value="14">Objectives object (14)</option>

  <option value="15">Objectives object (15)</option>

  <option value="16">Objectives object (16)</option>

  <option value="17">Objectives object (17)</option>

  <option value="18">Objectives object (18)</option>

  <option value="19">Objectives object (19)</option>

  <option value="20">Objectives object (20)</option>

  <option value="21">Objectives object (21)</option>

  <option value="22">Objectives object (22)</option>

  <option value="23">Objectives object (23)</option>

  <option value="24">Objectives object (24)</option>

  <option value="25">Objectives object (25)</option>

  <option value="26">Objectives object (26)</option>

  <option value="27">Objectives object (27)</option>

  <option value="28">Objectives object (28)</option>

  <option value="29">Objectives object (29)</option>

  <option value="30">Objectives object (30)</option>

  <option value="31">Objectives object (31)</option>

  <option value="32">Objectives object (32)</option>

  <option value="33">Objectives object (33)</option>

  <option value="34">Objectives object (34)</option>

  <option value="35">Objectives object (35)</option>

  <option value="36">Objectives object (36)</option>

  <option value="37">Objectives object (37)</option>

  <option value="38">Objectives object (38)</option>

  <option value="39">Objectives object (39)</option>

  <option value="40">Objectives object (40)</option>

  <option value="41">Objectives object (41)</option>

  <option value="42">Objectives object (42)</option>

  <option value="43">Objectives object (43)</option>

  <option value="44">Objectives object (44)</option>

  <option value="45">Objectives object (45)</option>

  <option value="46">Objectives object (46)</option>

  <option value="47">Objectives object (47)</option>

  <option value="48">Objectives object (48)</option>

  <option value="49">Objectives object (49)</option>

  <option value="50">Objectives object (50)</option>

  <option value="51">Objectives object (51)</option>

  <option value="52">Objectives object (52)</option>

  <option value="53">Objectives object (53)</option>

  <option value="54">Objectives object (54)</option>

  <option value="55">Objectives object (55)</option>

  <option value="56">Objectives object (56)</option>

  <option value="57">Objectives object (57)</option>

  <option value="58">Objectives object (58)</option>

  <option value="59">Objectives object (59)</option>

  <option value="60" selected>Objectives object (60)</option>

</select>
    
      <input type="hidden" name="row-6-9-id" value="60" id="id_row-6-9-id">

Yes, sorry - bad error on my part.

For the rest, I’m going to need to see the complete view and the definitions for ObjectiveTextFormSet.

No worries. I learned something figuring it out.

One question, in the FormSet, is extra=0 needed, I thought it was the default.

I really appreciate your help.

Here’s the form:

ObjectiveTextFormSet = modelformset_factory(ObjectiveText, fields = ["objective_text", "objective"], 
                                            extra = 0, widgets={'objective_text': Textarea(attrs={'cols': 15, 'rows': 5})} )

model:

class ObjectiveText(models.Model):
    user = models.ForeignKey(User, on_delete=models.CASCADE)
    objective = models.ForeignKey(Objectives, on_delete=models.CASCADE) 
    objective_text = models.CharField(max_length=1000, blank = True)
    timestamp= models.TimeField(auto_now = True)

view

def home(request):
    
    # Create unique prefixes for each element
                
    formset_list = [ 
        ObjectiveTextFormSet
                    (prefix=f'row-{i}',
                    queryset=ObjectiveText.objects.filter(
                        objective__objective_row=i
                        ).order_by('objective__objective_num')
                        )
        for i in range(1,7)       
    ]

    objheader = [1,2,3,4,5,6,7,8]

    # Add the formset to context dictionary
    context = {
        'formset_list': formset_list,
        'objheader': objheader,
    }

    if request.method == "POST":
        for formset in formset_list:
            if formset.is_valid():
                formset.save()
        return render(request, 'ISO22301/home.html', context)
    else:
        return render(request, "ISO22301/home.html", context)

Thinking about it some more, I don’t think that you want objective as a field in the form at all. I think the only field you need is objective_text, since your queryset is going to match the fields with the entities.

For debugging, you can print the errors in the formset in the else clause of the is_valid condition if you’re not rendering the errors in your form.

Regarding extra, default is 1 per Formsets | Django documentation | Django