Editable Form, from JSON

Hi folks,

Quick outline, I am building a small project using Django, (htmx and hyperscript, too) which will be a image metadata editor, and I’m looking for the best way of making a editable form from the data (for is pre-filled with data).

I am using Exiftool (a Perl built command line tool to extract exif data from images) to handle and read the data. Once extracted from an uploaded image the data is stored as Json, in a JSON field. I have successfully used a rather old module called django-split-json-widget which I had to update myself to get it working with the latest Django version.

Anyway, this enables my to do what I want to a certain extent, I can pass the data into my form with initial= and when the view handling the form is requested, I’m rending a editable form prefilled with the exif data. The only problem is that there is no control on how I can customise the form. It builds the fields from the keys, but you can’t expand on that and I can’t think of a way to do it. Without having to do a massive rethink on this whole project, would anyone have any input or point my in the direction of a package that could help? I looked at django-json-form but this is for admin forms only.

Also, the usage of json/json field is because I am extrating all the exif dta values, to quite a lot, too many to pass into individual fields.

It depends upon what you mean by “customizing the form.”

(You’ll find many examples here and elsewhere of people modifying form fields in the __init__ method of the form. That’s one of the easiest places to do stuff like this.)

What is it you’re looking to do here?

Hi Ken, I didn’t even consider overwriting the init , not really built any custom forms so still figuring this out.

At the moment django-split-json-widget produces the form and uses the keys from within the JSON passed to it, as form fields (fields displayed in client, I mean). I’d like to maybe add a description to those fields for example.

Interesting - I’ve never heard of that package before.

I’ve taken a quick look at it, and I’d be concerned about using it as-is. It’s old, has not seen any activity for years, and has this unresolved issue from almost 3 years ago: Python3 and Django 3 Compatibility · Issue #32 · abbasovalex/django-SplitJSONWidget-form · GitHub
with the first line of the description being:

This package is not currently compatible with the latest versions of Python or Django.

I’m not sure I’d rely upon this without forking it and ensuring it’s up-to-date.

For sure, I totally understand your concerns about using it, and I had them too. I’ve already forked the package and made updates, so it works with current Django. I guess the reasoning of my post is that I’d like to find another solution of achieving what I’m trying, in more robust way.

Cool!

I’m not sure I understand what you’re looking for in terms of “more robust”.

At its core, a Django Form is a Python class. Its purpose is to serve as a container for a set of fields, to render those fields as HTML, and to accept input to populate those fields.

There’s really little that you can do that is going to make it more or less “robust”.

Having said that, I can see the value in defining the JSON object as a MultiValueField, using the compress method to recreate the JSON value.

Having said that, that’s probably not the approach I would take.

I would pre-identify all the exif fields that I’d want to support and customize in some data structure, and create fields for them. I’d then add the needed fields to a blank form based on the data being supplied.

It’s probably a bit more work up front, but I think would provide the greatest flexibility.

Yea, I guess flexible rather than robust would be a better use of words!

Can you expand on this? When you say data structure are you talking about models? The exif data is made up over tens of individual values so I’d assume creating models to hold each value in a corresponding field wouldn’t be ideal. Hence using JSON as I want to support all Exif data key/value.

It’s going to be a lot easier at this point if I had some specifics to use. Can you identify 2 or 3 fields that I can use as examples?

As in fields I’d like in the form?

Yea, just a couple so that I can show some actual code working with data you might be looking for rather than trying to talk about this in the abstract.

Ok. So for example, Camera model, date, location.

Cool! that’ll work great.

The simplest data structure that can work here is a dict.

Example:

camera_fields = {
    'Camera model': {
        'type': forms.CharField,
        'params': {
            'max_length': 40,
            'help_text': 'Full camera model name'
        }
    },
    'date': {
        'type': forms.DateTimeField,
        'params': {
            'widget': forms.SplitDateTimeWidget
        }
    },
    'location': {
        'type': forms.CharField,
        'params': {
            'label': 'Picture location',
            'max_length': 50,
            'help_text': 'Location where the picture was taken'
        }
    }
}

You can then define a form as:

class CameraForm(forms.Form):
    def __init__(self, *args, **kwargs):
         super().__init__(*args, **kwargs)
         for k,v in camera_fields.items():
             self.fields[k] = v['type'](**v['params'])

However, in your case, let’s assume you only want a subset of fields, where the list of field names is built in the view. You can pass that list of names as a parameter when constructing the form.

Your form may then end up looking like this:

class CameraForm(forms.Form):
    def __init__(self, field_list, *args, **kwargs):
         super().__init__(*args, **kwargs)
         for field_name in field_list:
             self.fields[field_name] = camera_fields['field_name']['type'](**camera_fields['field_name']['params'])

(Yes, that’s the complete form definition. All fields are being dynamically created.)
If you run this in the shell, then printing the generated form is going to produce something like this:

<p>
    <label for="id_Camera model">Camera model:</label>
    <input type="text" name="Camera model" maxlength="40" required aria-describedby="id_Camera model_helptext" id="id_Camera model">
    <span class="helptext" id="id_Camera model_helptext">Full camera model name</span>
</p>
<p>
    <label>Date:</label>
    <input type="text" name="date_0" required id="id_date_0">
    <input type="text" name="date_1" required id="id_date_1">
</p>
<p>
    <label for="id_location">Picture location:</label>
    <input type="text" name="location" maxlength="50" required aria-describedby="id_location_helptext" id="id_location">
    <span class="helptext" id="id_location_helptext">Location where the picture was taken</span>
</p>

This handles the simple case. If you need to construct your fields in a way that this simple formatting does not allow (usually when you need to provide more complex widget definitions), then you can turn this camera_fields dict into a set of classes.

There are also a number of different ways where you can provide “overrides” for this on a per-instance basis, usually by passing those overrides in as additional parameters. This requires some changes to the __init__ method, but still doesn’t end up being too ugly.

(And yes, I’ve got at least four projects I can think of off-hand where I use this technique heavily - especially with Crispy Forms, since I can define the layout requirements for these fields and completely build out the form from a data structure.)

Ken, thank you so much for that it’s absolutely appreciated. I’ll take a closer look and be implementing your advice mid week when I’ve got more time, so I may have some more questions.

Awesome stuff thanks again.

Ok just sitting down to look at this. I’m just struggling to understand how I can make the dict in your post, I mean I have json stored in a JSONfield, like this (this is just a sample, there are over 100 keys in total)

{“SourceFile”: “foo”, “ExifToolVersion”: 12.77, “FileName”: “foo”: “foo”: “2.3 MB”, “Model”: “NIKON D850”, “XResolution”: 240, “YResolution”: 240, “ResolutionUnit”: “inches”, “Software”: “Adobe Photoshop Lightroom Classic 7.2 (Windows)”, “ModifyDate”: “2024:02:16 09:39:28”, “Artist”: “foo”, “Copyright”: foo", “ExposureTime”: “1/1600”, “FNumber”: 2.8,}

I the above methods are you saying I have to create a data structure all with the correct type of fields for each key? Sorry for another question!

Yes, if you want the degree of control that you’re saying you need.

No, if you want to define a default that is used for every key that isn’t otherwise defined.

Whether it’s 10, 100, or 1000 - that doesn’t matter.

The information you want to use must be defined somewhere. Whether you define them as fields in a form, or as an external data structure, they must still be defined. There’s no magic here that will do your work for you. If you need to handle 100+ keys, then you need to handle them and not just wish they didn’t exist.

Yea OK got you, thanks Ken, much appreciated, I’ll put some work into this, this week.

Ok I’ve done some work on this and now have more control over the form being created rather than using the Split JSON package. I’ve not yet added description for each form field, but, I think I’ll just make a dictionary with keys being field_name and value being a description (hopefully i can do some scraping and save typing each description manually!)

This is successfully rendering a prefilled form with editable data, and I am able to write the altered image data back to an image.

View.

#Form is passed a list, created from exif_data keys, and the image_exif_data (json)
form = ImageMetaDataForm(field_list=list(image_exif_data.keys()), json_data=image_exif_data)

Custom form

class ImageMetaDataForm(forms.Form):
    def __init__(self, field_list, json_data=None, *args, **kwargs):
        super().__init__(*args, **kwargs)
        if json_data:
            for field_name in field_list:
                initial_value = json_data.get(field_name, '') 
                self.fields[field_name] = forms.CharField(initial=initial_value, required=False)```

Ken, just reading through your post again, can you just give me an example of what a class would look like if I was constructing a form from classes? Cheers, just interested, this has been a real learning curve on building forms and has helped so much when it comes to more complex forms.

Constructing a form doesn’t change. What I was referring to was using a class to create the field definitions instead of a dict.

So instead of the dict for camera_fields like I defined above, you might create a class:

class CameraField(forms.CharField):
    def __init__(self, *args, **kwargs):
        ...

class DateField(forms.DateTimeField):
    def __init__(self, *args, **kwargs):
        ....

(The ... represents whatever code you want to use to create the field. You may need to do this because there could be situations where the constraints need to be different based upon some other characteristics.)

You would still have a camera_field dict:

camera_fields = {
    'Camera model': CameraField,
    'date': DateField,
...

Note that we’re identifying the class here, not making an instance of the class.

This means that the definition of the field in the form may look like this:
self.fields[field_name] = camera_fields['field_name']()

However, what this gives you is the ability to pass additional parameters to the __init__ method, allowing you to customize this at runtime.

It’s actually more work up front, but gives you complete programmatic control over the field.

Note: This can be extended even further by creative use of meta classes to allow these definitions to be stored in the database, making it possible to customize the forms from table entries if you don’t want all this specified in the code.

Thanks Ken. Absolute legend! Much appreciated.