Between where I define my models and where I consume them, I’ve found that everything actually cares about the QuerySet rather than the Model, but everything is designed to work on Models and cannot work with QuerySets IIUC. I’m wondering if anyone has run into something similar and has advice.
Here’s my specifics and thoughts.
I have an unusual design architecture from the get-go. I need a orthogonal plugin system, which is used to define and validate external scraping, data ingestion, and model shapes all in one spot. And I do have this – I’ll have to repeat this end plugin 5 to 10 times so I wanted something to make the plugins very small while enforcing a large number of constraints and adding in many needed aspects for free. One key result of this is that Models, Forms, Filters, Serializers, and a few Views are created entirely dynamically. This was acceptable until I needed the equivalent of an SQL VIEW due to needing two enforced and linked Models to be connected via a .select_related(). I need to plug this QuerySet into a Form with customized widgets (specifically the DualSelector widget) from django-formset. I have something to transform Models and QuerySets to Forms sort of, but it is definitely not working. I’m digging in more, but I don’t think I’m being particularly efficient in my work.
Herein lies what might be the first hard architectural fault I have with Django – that table data presentation is not abstracted from table data at rest. I can definitely be wrong. I could just be naive. But I feel like I need some guidance on how to get from a QuerySet to a renderable Form with customized widgets.
It would probably help if you showed some of your code where you’re having this problem, instead of describing it. It’s hard to give good advice, or understand the issue, without concrete examples.
Creating Models and Views dynamically is not something supported by Django. This tends to not work well in a real-life production deployment environment. (Or at least comes with a significant number of caveats and potential problem areas.)
On the other hand, creating (or modifying) Forms dynamically is something that is done quite regularly. (See the admin app for examples.)
You are not wrong, if you are referring to “presentation” as being how it is represented within the ORM. Django uses an ORM - an abstraction layer to convert table data to Python objects. It is not a “fault”, it is an intentional design decision.
If you need a different style of abstraction, then it’s quite possible that Django is not the best fit for you.
But as mentioned above, in the absence of actual specifics regarding the project, it’s next to impossible to offer tangible advice.
I gave a talk at DjangoCon Europe this year playfully titled “Dynamic models without dynamic models” because I realized – like you – what I wanted was not a dynamic model, but rather a factory for QuerySets with “smart annotations” that I could save back (and even register with the ORM to support custom lookups).
To have a prayer of interpreting the results of those smart annotations, I needed to remember how the data was fetched, so I ended up (ab)using the undocumented QuerySet._hints so that I had a place to retrieve this information when yielding my model instances.
Without hearing more about the patterns you’re using, I’m not sure how relevant this is, but I’m happy to go into more detail–I’ve iterated a bit further on this since the talk.
Here’s the jist, re-written for brevity. There shouldn’t be significant omissions for the purposes of this discussion, just for repeating patterns and for code related to more dynamic model usage. It is still large, and that’s probably hitting on several problems I have stemming from my inexperience.
I think there’s a few teachable moments bound up in here.
In my opinion, if you’re inexperienced and seeking advice, focus on writing the code instead of creating new abstraction layers.
One key result of this is that Models, Forms, Filters, Serializers, and a few Views are created entirely dynamically.
Django provides these abstraction layers intentionally. For example, some logic is best stored on a model, like a database model constraint, while a queryset is a great place to put complicated query logic. Likewise, a form class is a great place to validate the data sent from a browser for a post request.
If everything truly is a wrapper layer around a database schema, and you don’t plan on writing business logic in python, perhaps something like postgrest may be more appropriate.
trying to be helpful otherwise:
code that “generates” other code is generally called “meta programming”
A queryset has access to its model via the “model” attribute. Post.objects.all().model, which gives you access to the model’s _meta.
I would create a “django app” if I wanted to create a “django plugin”
descriptors are the magic for meta programming used in django