Best way to enforce a structure on a models.JSONField?

Using Django Using the ORM
1

Hello, I have a situation where using a models.JSONField seems like the best solution for my use case. I would like to enforce a schema or some other structure on the JSON, because I know it for now and would like to make the fields contents more clear. It would be best for it to be enforced every time data is written on the column in the database, and not to depend on endpoints to validate the structure.

My initial thought was to put validation logic in the model.clean function and then call it in the model.save. However I realized that save isn’t called when doing update or bulk_update, and so on. However something that is done on every write is encoding the dict. I could put the validation logic in a custom encoder, and pass the encoder as an argument to the JSONField. However, encoding is ran after clean, so I’m wondering if it is an anti-pattern.

I am thinking of creating a Django REST Framework serializer class for the JSONField only. I could use its validation methods to pass it a dict and it would format it correctly. Not sure if this is the best way to enforce the structure, I am open to suggestions.

What do you think?

2

bulk create request database directly.
if you want clean method before save, you use modelform or do full_clean method.

# bulk create example
a = [model list]
for i in a: a.full_clean()
bulk_create(*a)
3

Sorry, that has nothing to do with what I asked. Maybe you answered in the wrong thread?

4

Isn’t the question due to the clean method not working before bulk creation?

5

Welcome @nintskari !

That is a really interesting idea! (You could also create subclasses of the JSONField with your custom behavior “baked in”, which might “look” cleaner. I’m not saying it’s any better, it’s just an idea.)

It could be an anti-pattern, but, it may also be one of those things where you don’t really have another great option.

The only other solution that I could think of where you would be assured of having the structure tested would be the creation of a trigger in the database to perform the validation before being saved. (Think of it as a manually-created database constraint. Not saying it is that, but it’s a way to think about it. Again, just tossing out ideas.)

6

Great to hear some validating thoughts from you @KenWhitesell. I tried searching for an answer to this problem but could not find a good one. I tried the encoder approach out and it seems to work, but I actually might end up not using the JSONField after all. So I guess this is just food for thought :stuck_out_tongue:

You’re right, I considered utilizing the schema constraint feature in the database. It would be a lot faster than running the checks in python, but it would also be more challenging to make reliably. It would probably require at least these for good usability:

  1. Passing the json schema to a migration file from the field and generating the check in SQL, in a database specific way
  2. Logic to detect changes if the schema is changed so a migration would be required
  3. Validation to the schema itself
1 Like
7

Well, not really. I understand that clean is not ran for some operations and that it is intentional. I was looking for a generic way to catch all database write operations that create or update an object, and manipulate the data for JSON field specifically. But I think I was going a bit against the framework and will try to handle the situation without JSON field. Thanks

8

I’ve been through a similar situation, and depending on what command you use, there are methods that are not used.
The simplest way is to just add verification logic before saving.

1 Like