Proposal: New documentation page covering using Django with type hints

Django Internals Documentation
1

Hi all!

Only me here, just talking about how Django doesn’t support static analysis very well and that it doesn’t have many type hints. But, don’t worry, I’m not proposing that we add any! Pitchforks down.

This thread stems from a discussion with several community members in Athens, where I was arguing for the above, and eventually arrived at the conclusion that the actual problem I want to solve is not:

Django does not have any type hints

But rather it is:

Django does not interact well with typed code which is being statically analysed

And further to the above, not only does it obviously not interact well with typed code due to the lack of type hints, but critically: the developer experience of getting it to interact well is very poor and opaque.

I know how to make Django act like it is typed. And I am sure many here do also, but I do equally remember the first time I attempted this, and frankly, it wasn’t very pleasant, easy, or convenient to figure it all out, including ‘edge cases’ like custom Model fields, chained QuerySets, and custom model managers and custom QuerySets. And that was when I was approaching the problem as someone who is very confident with Python, and very familiar with Django.

A huge portion of Python users want to use type hints, or need to use type hints. It makes sense to me that we should support those people in doing so, and a very reasonable, low cost, low risk method of achieving that goal would be to produce a page, or section, of the documentation that covers the methods available for use:

  1. django-stubs
  2. mypy with the django-stubs plugin
  3. pyright
  4. PyCharm
  5. django-mantle
  6. ty - in the future

And provides an introduction to these things that is either:

  1. Reasonable enough that it encourages the user that Django can meet their needs in terms of integrating with a typed Python codebase
  2. Sufficiently explains why this tool is not appropriate for usage with Django, such as in the case of pyright and lack of QuerySet type inference.

Personally, I also think that it would be polite and wise to provide some sort of explanation (at a very high level) as to why Django is not typed, and why Django cannot easily be typed (at least, for a lot of it, anyway).

Obviously mentioning third party packages is an ‘approved’ thing to do now (although I’m unclear who approves these mentions), so it seems very reasonable that we should support users who want to statically type their code by telling them what packages are likely to work best, where to find their documentation, and why we haven’t just put it in the core yet.

I can draft this page if people think it is a good idea. Perhaps I would start as a blog post and get feedback and adapt.

If someone could clarify my query about approval of mentioning third party packages in the documentation also, I would appreciate it!

Andrew :slightly_smiling_face:

4 Likes
2

@anze3db’s recent blog post here has a good foundation for what we might say:

Pyrefly would need mentioning. And Zuban (from the author of Jedi) for that matter.

To quote Hynek, ty is “barely functional” — and its status is somewhat in doubt. I know Astral had all the momentum, but we really should see what happens now before recommending ty.

Yep. My recent DjangoCon Europe talk was precisely about this.

tl;dr Django is dynamic by design.

My particular conjecture here is that retrofitting typing is a lost cause/fool’s errand/etc, and so my approach is to build static islands on top — hence django-mantle.

But this is just a conjecture: a suitably herculean effort might show that it is possible. (At DjangoCon Europe Tim Bell — not on the forum? — was telling me that there’s an engineer at Kraken trying it [i.e. to fully type the ORM]; @hakib also wanted to see it actually typed, and muttered “it’s not that dynamic” at one point; and there are others — including you I think — who would like to see it done.)

Explaining that this is an open question, and where we’re at, wouldn’t hurt at all.

(FWIW: I also think we should be leaning into Python’s dynamic features, or we should all be using Rust, but that’s a different topic/aspect, and not for today :sweat_smile:)

So this would be a Topic page.

4 Likes
3

Thanks for the thoughtful reply Carlton.

I’m glad you’ve emphasised this point. To be clear, I agree 100%, probably should’ve left it out of the original post.

1 Like
4

I’d be happy to help out with getting the doc page set up using my blog post as a foundation (or not). Happy to open an initial pull request if/when we decide we want to go forward with this. I’ll keep an eye on this thread but feel free to mention me if I can help in any way.

2 Likes
5

@anze3db without wishing to prejudice any discussion that will happen below — if there was in fact consensus for it, and you are offering to do it, I’d personally be happy for you to do so, as I’d have to start from nothing :slight_smile: thanks!

1 Like
6

@carltongibson yeah, the main result of that work is found in the mypy plugin I made and the parts of our internal advice that we open sourced

The main source of tension is ultimately how long mypy takes to run (we have 30 seconds to startup Django for our multi million line codebase, plus between 1 to 5 minutes depending on how much of the mypy file cache gets invalidated). Fortunately it does appear that all the great work on mypy means we keep seeing meaningful performance improvements and mypy remains viable to what we do with it.

7

Oh, that’s great. I shall have a read. Thanks!

1 Like