Let’s say we want to create reusable code (Mixin) to be used by classes that inherit from Django’s models.
We want to ensure that the new class (our concrete model that inherit’s Django model) will implement required part of the API in order to use common functionality properly.
So is it a good idea to use those tools (ABC, abstractmethod, inheritance)
with Django’s models ?
WDYT ?
Hi!
What kind of logic do you need to implement?
Maybe what you need is to use abstract = True in your base model Meta. Docs
This is an example model you can inherit from if you want to include a created and modified field in your models.
Django Models are not simple classes like you might be used to working with. If you look at the source for the Django Model class, you’ll see it’s performing some heavy-duty metaclass programming. The class it generates is not a strict child class of the Model class.
As a result, the Django mechanism for abstracting functionality and fields into a base class is not as direct as specifying the base class in the class definition statement. See the docs on Abstract base classes for information on how to build common core classes extended by multiple concrete classes.
Lets say (totally imaginary example).
I want to create common functionalities:
class TranslatableMixin(django.db.models)
class Meta:
abstract = True
def translate():
...
class RevertableMixin(django.db.models)
class Meta:
abstract = True
def revert():
...
And i have model:
class Text(django.db.models, TranslatableMixin, RevertableMixin)
def get_locale(self):
... impl
Does that makes sense ?
Because, yes if trying to define
class TranslatableMixin(ABC)
@abstractmethod
def get_locale(self):
pass
def translate(self):
... impl
Things get mess, looks like exactly from that class meta programming Django stuff…
See the example in the docs referenced above. Also see the docs in the django-extensions project for an example of how functionality is made available across multiple models.
Another question is maybe it is not good at all to inherit all those functionalities but to compose as we know from ‘prefer composition over inheritance’…
The question how that goes with Django models …
Do you have maybe some beloved techniques in that field ?
Keep in mind that a model is a mapping of a relational database table to a Python object. I would categorize this as one of the impedance mismatches between the relational model and “pure Object Oriented Design”. This isn’t a object structure you’re creating here - you’re defining fields and functionality in the relational model of your data. That principle doesn’t apply here.
Yes, you could create a “true” composition structure in a relational database using OneToOne relationships, but from a database management perspective, that’s a sub-optimal solution.
The need is slightly different.
Django model represents concrete relational item as it will be stored in DB.
There is a difference between high level view on product feature or system component and it’s low level implementation parts. DB representation is very “implementation details”…What so called Data layer.
There are few arch concepts (DDD, 3 Layer arch, N Layer Arch, Clean Arch, Onion Arch). What is common for all those is the idea to separate the code base in a way it will reflect different perspective on the system. Business perspective for example is what feature/component/sub system should do in terms of product/end user/functional behaviour. Another perspective if how the relations in DB looks like, and that can be different from business perspective. For example We can have a concept of customer entity in terms of product, and we can have 5 different entities that will be used in order to achieve the functionality we need - lets say CustomerInfo/CustomerLocalizationConfig/CustomerStatuses etc…
Now the problem. In Django DB model represents before anything else Relational DB entity.
As well from what i saw until today it is very common to put as well business logic code into that model, combining feature related knowledge and concrete DB stuff related knowledge…
Without Django it would be separated mostly (DB for example will be hided by Repository class/pattern), business behaviour will have separate code layer (Customer Manager/Service), and for example workflows (CustomerSignUpWorkflow) that will orchestrate business use case…
Finalizing : This topic tries to understand how with Django approach keep models fat, and from what i see keep some business logic as well in models we still can share non DB oriented code on entites (DB model) that are DB oriented.
For another view on this topic, I’ll point you to a blog post written by James Bennett titled Against service layers in Django and the follow-up post More on service layers in Django.
If you don’t know who James is, he is a former member of the Django Core team, former member of the DSF Board of Directors, member of the Technical Board, author of Practical Django Projects, and one of probably less than a dozen people who really understand the ORM.
I point out his credentials because he has more solid, real-world experience with Django than I will ever acquire. He also has that ability to explain and teach very technical concepts in a way to make them easily understood - and realize he knows what he’s talking about. Those blog posts are worth your time to read and understand.
I still keep asking myself and not myself :
quote from the topic:
‘Django applications, the models — and potentially associated utility code, like custom Manager or QuerySet subclasses — are the API exposed to other code. Which in turn means that they are the place where the “business logic” should be implemented.’
Does the problem with that is when trying to follow that approach technical limitations is being raised immediately. For example we cannot use standart python’s ABC module with Django model… And i assume much more similar things… And i definitely understand why - we are trying to mix abstract (Approach, Layer, Design, code, System components) and concrete (Django’s ORM class for relational entities)
I am questioning myself if i need to give up on rich knowledge/practices (structuring the system in a "clean’ way) and standart tooling (same ABC for example)
for something else ? I cannot even define that else. Django’s way ? ORM’s way ?
And if we can combine those - i am searching for the way how to …
Never lose sight of the ultimate goal - produce a system that is cost effective and minimizes unnecessary costs.
All the solutions (architectures, design patterns, etc) proposed in the various books are “best practices” identified by authors from their experiences in the systems with which they have worked. Extrapolating those solutions to other realms is always conjecture.
Systems, environments, platforms and patterns all change. What was an ideal architecture in 1975 was not the ideal in 1985, and different again in 1995, and different again in 2005.
Architecturally, a “pattern” is always a solution within a given context. If the context changes, the architectural pattern may no longer apply. (It might, it might not - the skill of the architect lies in knowing the difference.)
Never ignore the ultimate goal. Period, full stop. If an architectural pattern can’t produce a quantifiable cost-reduction over the life cycle of that system, of what value is it? If I can produce and maintain a functional system in a framework for less cost than what it would take for me to do the same work using an implementation of some abstract set of patterns, then yes, I’m going to completely ignore those patterns.
My pet phrase for this is “Don’t fight the framework.” Django’s architecture has evolved to where it is today because it works. And at the end of the day, that’s all that really matters.
@KenWhitesell
Agree, specifically with the “ultimate goal”. That is undisputed fact.
But can i focus on some things you mentioned and to add perspective ?
For example:
“Django’s architecture has evolved to where it is today” - good statement but it requires as well the reasons why Django’s architecture has evolved to where it is today.
My question here is what is the main enabler of some group of people when they choose Django ? Does the same enabler is there after 4 - 5 years ? Is it easy to replace it ? Maybe yes, probably no.
Why “Django’s architecture has evolved to where it is today” ? Because it answers short lived decisions ? long ones ? It reacts to the consequence of short lived decision (Like ok now we are not already startup but we still cannot afford rewrite what are we do) ?
What i am actually asking is what kind of systems/people/companies are the Django user, how does that changes with the company’s change and how that consumer influences the product itself - Django?
Cause and effect relationships - can it be that Django does not addresses business logic layer not because it do not want to but from the reason that it did not initially and it is already too far to try even…?
Sharing my thought , not making statements
Specifically my work.
It is 5 years old Django codebase, having both REST api (rest-framework) and Admin part.
Both parts operating on the same models and features but with sometime very close, sometimes not so close workflows.
So the questions of code duplication, difficulty, easy navigation in the code base are very relevant.
No any layers exists in the system. All done in framework parts, and the relational DB models are not reflecting the business perspective in a way that allows efficient communication (DDD).
A lot of problems/bugs/misunderstandings with that… Product/business do not uses Django’s ORM models for his view of system, and he should not. So de facto there are workflows, features and entities that the business defines and operates with but they does not completely reflected via ORM models, and that make sense. We do not want to create DB table only to reflect business language without real need for it… Yes we want to create code entities that reflect business language and it is not Form/View or Model…Django does have appropriate part in my opinion for that but who said it should ? It is only python finally no ?
Would be an interesting study, yes. But personally, I wouldn’t go so far as to say it’s “required”. I’m enough of a pragmatist to acknowledge that “it is what it is” - I can use it (or not) regardless of how we got here.
The investment in any sizable web framework is huge. Any company choosing to build core logic around .Net, Spring, Django, Symfony - or their corresponding “products” such as Liferay, Wagtail, Drupal, Wordpress, etc has typically made a decision that they need to feel good about for 10 years. Replacing any such product is never trivial. The frameworks, in providing the shell in which the business logic resides, does drive the architecture.
I’ve only been using Django consistently for 7 years now. I know, empirically, I can deliver solutions faster, easier to extend, and with fewer problems across the board than I can with .Net (4 years), Java/Spring (8 years total), or Drupal (5 years). (There is some overlap there, those Java projects were concurrent with both some of the .Net and Django work. Only about 3 years of that was exclusive to Java.)
I’ve worked with 3 large Java projects - two of the three exhibited a bad case of “over-engineered layers”. (The third had no discernable architecture at all - it was a complete mess.)
One had all the code nicely laid out between servlets, data access layer, business logic layers, and JSPs. And it was probably the worst system I ever had to deal with. I wish I had permission to keep a copy of it. It adhered to every generalized “best practices” pattern of Java engineering you could name. Yet it was also the most difficult code I ever had to dive in to to try and diagnose problems, or to extend functionality. I’ll take the most disorganized Django-based project over that one any day of the week.
My opinion is that it’s not needed independent of the models and managers. Such a layer, as a separate entity, provides no value in Django.
For a real-life example:
I have a system I’m working on now - part of that system is a Timesheet reporting system. People enter their time every week, reporting hours by day by project and task.
A “business entity”, the Timesheet, actually consists of data from six (seven? I might be missing one) different tables. Each of those tables are separate models. However, most of them have no intrinsic existence outside the context of that Timesheet.
The base Timesheet object itself is rather small. It consists of an id field (primary key), a foreign key to User, creation date, last modified data, week-ending date, and a status. Each row of the timesheet (a separate “Timesheet Row” is the hours for a specific task) is related to it, and each day (a separate “Timesheet Hour” table) is related to the Timesheet Row.
In this particular case, the “public API” is the retrieval of an individual timesheet. That functionality - retrieval and organization of all the data associated with the timesheet, exists within the Timesheet Manager and the Timesheet Model.
Why? Because in this case, the “Logical timesheet” - the “business entity” is fundamentally driven by the Timesheet table. There is no reason to put that logic anywhere else. Creating a separate layer, just to retrieve that object and its related data, is adding complexity without adding value.
With that pattern in mind, our database in this system consists of 6 base entities - business objects such as Users, Work, Invoices, Timesheets, Reports, and Firms. Each one of those is a table in the database, but has multiple tables (and relationship tables) supporting them. (The current database is about 125 tables total.) Those business entities serve as what might be construed as the “business layer” in a different framework - they certainly provide the same functionality, but without the artificial segregation of code.
If I’m having an issue with a User’s permissions, I know that all the logic associated with the permission system is in the User module.
So the point here isn’t that there’s no “business perspective” on the data - there clearly is. The point is that there’s no value in creating a separate architectural layer for it. The managers and model methods reflect both the logical views of the models themselves and the business views “anchored” by that model. And that is my understanding of what is meant by the “Fat Model”.
As a side note, to try and provide some balance here, I searched for some of the blog posts I’ve seen in the past that argue against the “Fat Model” idea. However, the first 4 that I found actually agree with me, in that they take the position that the logic belongs in the managers not in the models themselves. I’ll accept the argument that I’m inaccurate by including the managers and querysets as being logical components of the models. Personally, I include them there since you access those managers through the model’s class name, and when I refer to the concept of a “Fat Model”, that includes those related managers and querysets.
@KenWhitesell
Thanks for adding real examples, that gives a new perspective on all the theoretical discussion.
I see from your examples and from all information I received that it is ok to put business logic into models and managers, it is still essential to create common base between business vision and relational entities that represented by Django models, and that can be challenging for somebody less experienced than you.
Because if there is physical boundaries in code, for example business layer, and developers should obey the restriction by focusing on modeling business around abstract entities and not dive immediately into implementation, messing thing, then those restrictions will secure developers from mistakes more than if there is no such boundaries.
I’m my opinion not creating those abstraction layers can suit some dev teams and cannot other.
WDYT?
My opinions here are more philosophical than necessarily “practical”. But they are well steeped in history.
Python and Java have two distinctly different design philosophies.
Java, both the language and the JVM runtime, are engineered to allow a library to “protect” itself - hide implementation details and prevent access to data elements. Its very strict access rules exist to allow for the construction of a system that can minimize the “damage” that a poorly-written component can do to the system overall.
Python, on the other hand, is designed to be open. There is no way a class can truly hide implementation details or prevent access to data attributes or even to prevent runtime alteration of a class. The whole idea of python is that it provides all possible power to the programmer.
For more on this, see the section We are all responsible users.
So, how does this apply here?
Simple, there are two points to make. One is practical, the other, philosophical.
First, the practical - any “restrictions” you think you’re applying are only conventions. There is nothing within Python that allows you to enforce that type of structural restrictions.
What you have then is a management problem either way. Someone needs to stay on top of the code being written, so why not choose the easiest way to avoid judgement calls and arbitrary decisions? Why add the cognitive load of figuring out whether a function belongs “here” or “there”? It’s a lot more obvious that (to borrow from my earlier example) any function working with a Timesheet business object belongs in a manager. I don’t need to make a decision as to where it will reside - and if a particular function changes in scope, there’s never a need to consider whether it needs to be relocated.
The other point is more philosophical - Python is explicitly designed without the “guard rails and protective walls” provided by some other languages and runtimes. The phrase used to describe this idea is “we are all responsible users”. Since the language doesn’t provide those protections, the responsibility resides on the programmers to, as a former manager of mine used to say, “don’t do stupid stuff”.
So yes I would fall on the side that, if you’re building a large enterprise-scale system that you want to be cost effective, you need enough experienced and knowledgeable developers to keep everything on track, with the authority and responsibility to review and correct mistakes along the way. If you can’t do that, then I would suggest you might want to consider a different platform for your project.
(And to be clear, I’m not saying this should be considered a requirement for all projects. I made an earlier comment basically saying that a 100-line system doesn’t need any such oversight, a 1000-line system should have some, but a 10,000-line system absolutely requires experienced developer involvement.)