Where to put business logic in Django?

Using Django
1

I’m curious if there’s any consensus here. Obvious answers are:

  • Models/Managers - make them FAT is often cited as the default but can have performance issues and in large projects become unwieldy to reason about
  • Views - obvious place to put them but can also grow too large and not performant, plus do you really want all the calculations done on the fly?
  • Templates - not advised to put much logic in here, bad for performance, hard to reason about
  • Separate Python file…this is what I see larger sites doing. What they call the file varies but it sits between models and views and handles outside API calls, extensive logic, etc. Not aware that this pattern is codified anywhere.

Curious what others think and if I’ve missed anything here :slight_smile:

9 Likes
2

So I’ve tried each of your bullet points and encountered problems with each of them. The same problems you’ve mentioned. I worked with my current company to codify a styleguide that is similar to your final bullet point - putting application logic into a separate layer we called “services” that ties together models and views. You can have a read of the guide here.
One of the issues we had with all the other options was tight coupling. We kept struggling to separate business logic, database representation, and API representation. Sticking the service layer in between made it much easier for us to maintain this separation.

I don’t think it will work for all cases, but it has worked well for us. Let me know what you think.

7 Likes
3

Cool guide/repo! This is exactly what EdX does, for example. And I suspect others of similar size.

1 Like
4

I’ve also used the “services” type approach - basically a “fat model layer” but not in the model layer - with quite a bit of success. You have to make sure you keep good separation, but that tends to be easier with them pulled out like that.

At my current work we have a full-blown [micro]services infrastructure, which is less recommended unless you’re big enough to maintain something that size.

3 Likes
5

Hi, this is indeed a common problem to deal with. And it’s not even Django or Python-specific.

There are some good examples of architecture approaches solving this coupling of logic to the MVC frameworks.

For example, you could take a look at Clean Architecture by Robert Martin, or Hexagonal Architecture by Alistair Cockburn.

These are relatively complex approaches if we compare them to the classic Django.

They take a decent amount of time to implement. They require you to think about a problem you’re solving in terms of the domain model. They require you to do decomposition properly.

We want to lower the entry threshold for that technique. So we decided to start the dry-python project to give programmers a set of tools to express business logic easily.

You can take a look at the tutorials repository to have an idea of how that all feet together.

Best regards,
Artem.

6 Likes
6

Phoenix explicitly encourages an internal service-like layer with Contexts. I really like this approach in theory, though I haven’t written anything serious enough with Phoenix to pressure test it. As a side note, I also like how the web/REST interface itself is treated as a separate Context in Phoenix.

At edX, we’ve done some work around having service interfaces declared in an api.py file to give a service layer, but I keep wanting to reach for a better solution. I just read the docs for django-api-domains linked above, and I broadly agree with the high level concepts of having an explicit service layer, but I feel like every time I’ve tried to walk down that path in my own attempts, I end up with something clunky and redundant. Please understand that I think Django is great and I believe that Open edX owes much of its success to the framework. I think it’s very reasonably put together. It’s just that I can’t line up all the pieces in a natural way in my head for this use case.

If I could magically have everything I wanted, our codebase might look like:

  • ~10 domains, each having potentially ~10 apps inside (total codebase ~1M lines)
  • Each domain has multiple public in-process service interfaces (one per named release), to make upgrade cycles for third parties easier.
  • Each domain has some set of data structures, and signals that can be hooked into.
  • There exists some straightforward pattern for third parties to extend by associating their own data with something in the domain. For instance, say my custom app wants to be able to add something to a course enrollment, so I can pull that up without going through N+1 queries.
  • REST or GraphQL APIs could span multiple domains, and talk to them via their service interface.
  • We should leverage existing work as much as possible–I don’t want to introduce something that looks alien to other Django devs or that requires a lot of work to maintain.
  • Django Admin might be nice, at least for simple operations, but I could live without it.

My goal would be to consolidate business logic and maintain a smaller, more stable public API for other domains and third party apps. Hiding Django (“the framework is an implementation detail” philosophy) would not be a goal in and of itself. I want to add as little as possible to achieve those goals.

But here I start running into some issues:

Where does validation happen?

We don’t really use Django forms, preferring to use React frontends and DRF for the REST APIs they talk to. Put it in the model’s save()? That’s less than ideal, and hard to bubble up the exact errors. DRF has some nice validation/serialization facilities, but that’s supposed to be at a layer outside the domain’s service. Make something in the domain service’s public API to do validation that happens to trivially convert to something DRF can use?

Maybe the REST API layer is really a peer to the service layer and not a layer on top of it, and so has access to all the internals? But how would cross-domain APIs work? And would we start to see drift in validation/logic rules between the REST API and service API?

Hiding the models

We don’t want to pass models around through the public service interface because we want more freedom to change how we store the data behind the service, and we also don’t want to pass around a giant API window that allows people to arbitrarily query the database in completely unpredictable ways.

So maybe data classes, with translation happening at the service layer? But again, DRF has all this nice stuff that expects to work with models. I mean, basically everything in the Django ecosystem has nice things that work with models directly because they expect to be the interface between models and the browser making requests. So I can see how it would work, but it feels a like you have to leave a lot of conveniences to get there.

Efficient Querying

N+1 queries are one of the banes of my existence. But avoiding them means some level of model relationships. Possibly create a limited API here that allows you to key off of the primary key of another Domain’s model? This sets off all kinds of coupling alarm bells, but there are both performance and data integrity arguments for having something at this layer. This could probably be limited to only one or two models per domain, for the most common things that are hung off them (e.g. courses, enrollments, students).

I’m working on a little side project to experiment with these concepts. I’ll definitely be looking at django-api-domains and dry-python while I’m exploring ideas. Thanks folks!

1 Like
7

At my workplace, Carbon Co-op, we have a 10-month-old Django monolith, which I broke ground on, and so I’ve been looking for other people’s approaches to this too. None of our engineering team (4 people) are super experienced with Django. I’m probably the most experienced and I’ve been working with it for about 2 years.

We’re not a big organisation and part of using a Python/Django/AWS stack is to use battle-tested and well-understood frameworks (basically the opposite of niche). We’re using Django in a very batteries-included way: we use crispy forms to create UIs and all that. So something like your API Domains guide @phalt is really inspiring but it doesn’t really fit for us.

dry-python is also a really interesting project, and it’s cool to see so many concepts from strongly-typed functional languages put into Python. But it’s a radically different way of writing code that isn’t that common in the Python or Django worlds so it’s a hard sell.

So really I’ve been looking for something that implements a “lite” version of clean architecture / hexagonal architecture, that gets you a good amount of the benefits without fundamentally rearchitecting your Django code and thus losing its anti-niche / batteries included status. The best thing I’ve found for this is from Hack Software - first I found their style guide and it turns out they also did a talk at EuroPython about it too. I’d recommend taking a look - the talk is a bit more informative than the styleguide taken by itself.

I imagine this sort of stuff really varies depending on the size, maturity and scale of your organisation, so YMMV - and probably will!

5 Likes
8

At my company, we’re three developers maintaining a codebase that has been developed “Fat models” style.

We’re currently refactoring our codebase to an architecture resembling the Clean Architecture, and though it is a lot of work we’re already seeing benefits in terms of testing.

Our platform is basically a dashboard with financial data, the model domain is pretty complex and there is not a lot of CRUD functionality available (most use cases are reads + lots of calculations on the fly). In cases like these I think services / use cases layer approach works great.

I have a bunch of links that I’d like to share to resources we’ve used to inform our decisions, but I can only post two since I’m a new user:

Django model Guideline
Business Logic in Django projects

I think this is a very interesting topic and I’d be glad to discuss more details regarding our own approach if anyone is interested. I’m also looking forward to reading Harry Percival’s book on architecture which can be found on github. If anyone is interested in the rest of the links to other articles and talks about architecture just get in touch and I’ll send them.

3 Likes
9

Hi, i am a new developmenting in django Framework. And i have a question, where must i put the factory code to generate data for models and database With Factory_boy and Faker. also, I wan to Know how run this factory in the Shell including the Imports commands. Thanks a lot

10

thanks for the props Cristóbal!

for anyone curious, the architecture book in question may be found here, https://github.com/cosmicpython/book, altho the short answer is “if you’re doing this sort of architecture, probably don’t use django”. the reasoning is that, if your domain is complex enough to warrant formal DDD / Domain Modelling, then it’s no longer a CRUD app, and you’re outside the Django sweet spot really.

of course not everyone starts with a green field, so we’re hoping to build out some pointers in the final chapter for how to do these sorts of things when starting from a probably-django monolith, but they’re not fleshed out atm.

there is an appendix on “how to do Repository Pattern, Unit of Work Pattern, and Service Layer Pattern with django” – ie how to fully decouple your model from the Django ORM. it’s painstaking, but not complex.

3 Likes
11

Over the years, my practices settled down to this:

  • For form views, I put most of the business logic in the forms.
  • Some of the business logic is set in model methods and managers.
  • Some business logic is set in the background tasks (Celery or Huey).
  • Some business logic is set in the management commands and executed via cron jobs.
  • Templates have minimal logic, like what to show or hide based on different conditions.
12

Hello, I am new to using the services layer and I support your opinion, but I would like to know how to deal when a service requires many validations, 10 for example. From your point of view what will you do? Would you perform the validations on the service or invoke methods in the service itself?
I appreciate your response

13

I’ve been mulling many of these points with respect to my projects and here are my thoughts:

  1. Fat models are okay at the start, but seem to become a problem the more the app grows
  2. A separate module for “business logic” is probably a good idea
  3. I think models are a “low level” interface to your data(base). And should not be much more.
  4. I call the business logic module “logic”. Others seem to call it “services”.
  5. When you need to implement a process that writes to one model then I think a manager is the way to go.
  6. When you need to implement a process that needs to write to two or more models, then I think you might want to use the “logic” module
  7. If you want to read any data that depends only on one model, then you can use model methods
  8. If you want to read data that depends on more than one model, you might want to use the “logic” module. The exception to this I think would be when a model is conceptually “nested” in another (via a
    foreign key) then it may be natural to use model methods.
  9. The logic can be reused in views, api calls, celery tasks / cron jobs, management commands
  10. The logic module is a natural interface to your app.
  11. Forms are also a natural interface to your app. However I would keep form logic simple, and if it gets complicated (e.g. requires writing to two or more models) then factor that logic out into your logic module.

Hope this helps someone!

5 Likes
14

I think this is a very good summation. It rings true to me, at least, though I think a developer needs to suffer through trying various different approaches to see the wisdom in this.

1 Like