Django needs a REST story

Django Internals
21

This is just not true for a whole variety of web applications. The reality of the modern web is that you do need REST if you’re using any amount of frontend framework, a mobile app, or have more than one service.

This is the best illustration of the necessity of good REST support. If people are using FastAPI over Django just for its REST/serialization layer, that indicates that Django is severely lacking in this department.

The Django community page already recommends both DRF and Django Ninja, and I believe that option 3 (have more on these in the Django docs) should be about both and why you would pick one over the other.

I personally recognize the age and challenges of DRF. I also think Django Ninja is too “FastAPI-like” in how it does its views. I’d personally prefer to use Pydantic based serializes with either DRF style views or Django Forms style class-based views, but that’s my personal preference.

Preferences vary, which is why I believe Django should build it’s own solution long-term, and that solution should be minimum viable infrastructure that is modular (allowing users to easily replace parts with community or homebrew alternatives). I think Django shipping API views with a modular serializer layer would be a great initial version of this feature, especially if these views were already compatible (or nearly compatible) with DRF and Django Ninja serializers.

2 Likes
22

+1 for Option #1.

I’m not a fan of DRF’s Rails inspired APIs and features. They’re awkward. I’d much prefer the Django project do its thing and create the kinds of Pythonic/Djangonic apis that they’ve been making for the past 20 years. This is a simple problem that needs a simple solution.

23

This is the best illustration of the necessity of good REST support. If people are using FastAPI over Django just for its REST/serialization layer, that indicates that Django is severely lacking in this department.

What I meant was that FastAPI has probably been a breath of fresh air for anybody forced to work in DRF, which has been detrimental to the continued uptake of Django for new projects that involve REST. Recommending DRF more strongly, or integrating its philosophy directly into Django, would be a bad idea (that’s the non sequitur I was referring to). I’ve worked with at least one person who, if Django Ninja existed one year earlier, they would have never switched to FastAPI.

I’d personally prefer to use Pydantic based serializes with either DRF style views or Django Forms style class-based views, but that’s my personal preference.

I definitely agree with the Pydantic part and the latter (CBV) part of this.

I also think Django Ninja is too “FastAPI-like” in how it does its views.

I also agree with this, to a certain degree.

With all that said, if I had to pick between the way DRF does things and physically mailing envelopes with printed JSON, to each API user, I’d pick the latter.

I think some of the primitives can be put into Django without being too opinionated (aside from maybe supporting Pydantic), but I think any attempt to do so should start with option 1, sub-task 1, by simply providing the validation and serialization, and perhaps some small foundational work for sub-task 2 (CBV).

3 Likes
24

i agree. it’s also very hard to compare the developer experience of the drf view layer to the fast api view layer. if we’re building an internal app that’s not going to care about content negotiation, permissions, throttling, etc, I could sympathize with the feel of bloat. however, it was pretty nice the one time i needed to support message pack along with json. or the few times i’ve needed to support form data as well as json. that is part of the “batteries” that i tend to appreciate over time.

The issuewith DRF is that the serializers arelast generation.

besides slow, what specifically does last generation mean? do you mostly mean the developer experiences that come from using runtime type hints?

25

I meant only that they’re slow.

Look at the benchmarks on the Django Ninja site. One is nonsense: it’s comparing async views to sync views with a blocking IO operation. The whole point of async IO is that async will process more requests in this kind of case. The other though is showing the contrast between older and more modern serialisation. There are significant speed ups to be had.

There’s a nice talk from PyCon Italia by Tin Tvrtkovic, the author of cattrs that explains what’s going on here:

The typing issue is totally separate. This gets into the weeds very quickly: I would hope not to derail this conversation by a tangent into typing.

Aborted musings…

For simple cases type hints are lovely:

from dataclasses import dataclass


@dataclass
class MyDTO:
     name: str
     count: int

Wonderful.

When it gets more complex though, needing validators and all the rest of it, that we pass to a field, the examples aren’t any different in kind from Django’s forms.CharField(...) &co that we have. IRL these more complex cases are often the norm. As, then, is often the case, the actual complexity of the problem means that the great simplest-case demos don’t really continue to deliver. I look at many examples using Annotated and realise we live in different worlds.

There are DX gains to be had from any fresh API design — learning the lessons of history. Type hint support (for those simple cases) would be great. But it’s not the New Jerusalem.

As I said, this isn’t a conversation for this thread.

26

Not in favour of option two. DRF is massively complex and over engineered with its too many layers. It is only suitable if your API works exactly like the model does and if it doesn’t, it becomes increasingly difficult to implement. I find DRF to have a very flat learning curve. If something gets put into core (which it should), it shouldn’t be DRF in my opinion.

27

Django has everything you need for REST. I talk about this in my talks “µ-Django“ or in my workshops “Build REST API with raw Django“

The problem is, that all of this is poorly documented. For example, I see that many people still misuse the terms “serializer” and “validator.”

Django has an awesome internal part for serializers. It works much more flexibly than Pydantic because Pydantic mixes data serialization and validation tasks.
Django has an awesome internal part for validation (forms). It works faster than Pydantic v1. And forms are still faster than Pydantic v2 if we use complex business validators. I explain how this is possible in my “Django-FTL” talks/workshops.

Django has an awesome GCBV, which works as a REST-API view together with JsonResponse. I use TemplateView + “response_class = JsonResponse”. The problem is that TemplateResponse breaks the standard order of arguments during initialization and expects a “template” on the first place, not data, like other xxResponse in Django. I solve this problem by overriding __init__ in the jsonresponse class. Yes, this can be improved.

Django has another problem with the latest Error out-middleware. It also needs to be updated: if we see content-type json in the request, the response should not send back html, as debug.technical_404_response does. I also show how this works in my workshops.

API documentation is handled with two lines of code, using the documentation-first paradigm (schema.yaml). I don’t need to change Django for this. However, I change the method OPTIONS from django.views.generic.base.View, which in Django does not allow a body at all, and I send Yaml/Json in the body according to the headers with documentation for the current URL. Something similar can be found in DRF, and I like that approach.

By the way, i generate Django models from shema.yaml automatically with codegen, although with the latest AI-agents, this is not important at all.

In this case, to add REST API support to Django, you only need to create documentation without adding any additional elements to Django.

Or @zags do you want to add additional elements in Django that will duplicate existing parts in Django?

3 Likes
28

@danilovmy thanks for the perspective. Is there an example project we can review with these concepts? I watched your talk on uDjango, and while it is great, it doesn’t really touch on most of the concepts you present in your response here around how to build an API-first application using Django. It is more about breaking up an existing Django application, where things like migrations, models, and logic are stored already, into new, small, isolated endpoints. I would enjoy reviewing an example application that might apply some of the concepts you discuss here in a concrete fashion.

I’m curious how you use the Django serializers without coupling them to your model layer. I definitely haven’t used this layer much. My team often finds that the shape of our APIs and our data models end up diverging in a domain-rich application, so efficiently and explicitly defining our API layer is often important separate from our models when building APIs. Are there ways to do this with the built-in serializers?

I generally don’t build my APIs as schema-first, especially for APIs built for the frontend engineers within my own company, because I communicate with my frontend team in other ways. This is usually through typescript interfaces, which we find easier to write and more expressive in iterative design phases. However, having the documentation generated for us after the fact (either with DRF or Ninja) is a quietly awesome feature once we reach maintenance mode and want to review what features are supported by our system - all of those forgotten fields we built a year ago that we might revisit. My past experiences with managing YAML schema files manually was painful. It has admittedly been a few years since I tried, so maybe the tooling is better now. Within a larger application, one with hundreds of routes, even if aggregated across multiple modular services, do you have techniques for managing these schemas manually?

I’m also interested in your patterns around JsonResponse and the template views. I have used basic JsonResponses in the past, especially when absolute performance was important. At times like this, my team has used either plain dict or TypedDict and passed the data straight to a JSON encoder to get the information back out to the requester as quickly as possible. I’m curious if your approach is similar.

Side note: I agree with many of the statements here about putting too much validation into an API layer. I think it is best to keep it simple. Validate shapes and type of data. Maybe toss in some regular expressions. As soon as our team started digging into relationship validation, we found we were really hitting domain logic, and we prefer to pull that out of our view layer entirely.

2 Likes
29

@zags I super super appreciate you making this post! As @carltongibson notes we’ve been discussing this for probably around a decade, and you’ve done an excellent job summarizing those past discussions and laying out the options clearly.

I share your feeling that Django needs a “story” – we can’t keep having the same circular conversations for another decade! My hope is that this conversation you’ve kicked off can eventually turn into a DEP (several DEPS?), and then a concrete decision and plan going forward. I doubt that we’ll find consensus here, so this feels very much like a situation where “disagree, then commit” should be seen as the optimal outcome.

I’m happy to help author or edit a DEP or two here. I don’t want to be the primary author but I am happy to support, including doing the heavy lifting on the writing parts since I know that’s intimidating. Feel free to hit me up.

1 Like