Combining Urls, Views, Forms, and Models into fewer files

Using Django Getting Started
1

Greetings everyone,

I watched @carltongibson give a talk at DjangoCon Europe on YouTube called Yak-shaving to Where the Puck is Going to Be. In his talk, Carlton talked about adding views into urls.py. At the end of the talk, he also advocated for adding forms next to the views they support.

I’m curious how many people ascribe to Carlton’s notion of combining some (or all) of the normal app modules (urls, forms, models, views, etc.) into fewer files (or even one file)? The primary benefit being locality of behavior and not having quite so many files open at once. While you get used to it, I rather dislike having three or more files open just to make some relatively simple changes and having related things scattered across files. Having one or two files open most of the time sounds appealing and could reduce cognitive burden while coding.

Carrying this line of thinking to an extreme, one might organize apps into components (probably) around a model or two. For example, if you were creating a large newspaper project with an authors app, you might have an authors.py file that has all the forms, views, and possibly even models all pertaining to authors. Inside the same app, you might have an awards.py to keep track of all the rewards your authors have won over the years that is similarly self-contained. Maybe not the best example, but hopefully you get the idea. This approach is conceptually similar to how VueJS organizes it’s Single-File Components (i.e., the HTML, CSS, and JS are be in the same file), which people really seem to enjoy. Note, the above is just one possible (and purposefully extreme) example to explore the ideas that Carlton mentioned.

Is anyone else combining the modules in their app(s) into a single file? If so, how are you structuring your modules? I don’t have any experience developing medium or large size Django apps, these are just ideas I’d like to explore for an app I’m writing.

2

<Conjecture> Anyone who thinks that Django spreads too much across too many files has never worked on a large enterprise-scale Java project. </Conjecture>

I’m of mixed feelings about this, and I guess in our case it would depend upon the project.

And now that I think about it, that statement alone is the only reason I need to want to say “no”.

You talk about the cognitive load of opening multiple files - I think you need to weigh that against the similar load of remembering “Where is “Form X” in this project?”

Our team supports something in the neighborhood of two dozen different Django projects. Only two or three are of any substantial size - but on any given day, I may need to work on any of them.
For us, we are in agreement that consistency among projects has value by itself. It reduces the friction between tasks if we can expect to know where to look for something.

If I can’t identify a common pattern for organizing code across the entire library, I’m not going to want to adopt it. One of the advantages that I have found using Django is that we can support those organizational patterns with both large and small projects.

Part of this also comes from the “one app until it hurts” philosophy. Our “components”, such as they exist, never work with less than 20ish models. We don’t even begin to think about trying to organize into apps until either the models.py file exceeds 2000 lines or we have a clear and unambiguous knowledge that some type of functionality is going to be useful across multiple projects.

If I were to take our two or three largest projects out of the picture, my attitude might change in some limited areas.

I could see including forms in with the views - but that’s about it.

I would see no value in combining the models with the views - the needs for organization are completely different between the two. Our experience is that the models more closely interrelate with each other rather than with any specific view.

URLs? Leave them alone. My tasks are generally given to me in the context of “make such-and-such change to the page at /some/url”. The urls.py file is my roadmap for getting started in that case - I don’t want to go chasing them down.

admin.py? The admin is a separate app. It would need to be changed to allow for the admin definitions to be loaded from the models.py file.

Templates? I’m not sure there’s any value in trying to combine them with anything else. (Having said that, we do have some cases where we render template fragments from strings for HTMX-based systems. The template strings are stored in the views using them.)

1 Like
3

Hi @Jason.

Thanks for watching the talk. I hope you enjoyed it.

I wouldn’t want you to read too much into the points I made there: as I stated/disclaimed several times, the talk concerned what I’d been up to, and the L.o.B. points are more relevant when your code is new and in flux.

As a project grows you need to move code into the right place™, so that you can (continue to) find it. I think the way we’ve done it is ≈right. Put your forms in a forms.py, and so on.

Reading @KenWhitesell’s reply, I guess that’s the point he’s making too, and I agree – if you’ve got a lot of code you need to be organised. :100:

Nonetheless, an awful lot of coding goes on in circumstances where keeping things together is helpful, and does help you go faster. (A lot of it is the young, and still in flux that I was specifically addressing.) I wrote a recent piece in The Single Folder Layout for Django Projects that touches on just this same point in another way. (Someone asked on the fediverse about single file applications, which you can do, but you pretty much always want the extra structure a proper project gives you… Link)

On the single file component idea, take a look at Tetra, by @samwillis, which combines Django and Alpine.js to do the kind of thing you’re talking about (I think), as well as Django-Unicorn by @adamghill, which is not unrelated.

Like all these things, balance is key. (If you do go mad, please don’t say that I told you to do so :stuck_out_tongue_winking_eye:)

Kind Regards,

Carlton

1 Like
4

Ken, thank you for your (as always) thoughtful and though provoking reply. You’ve given me a lot to think about.

<Conjecture> Anyone who thinks that Django spreads too much across too many files has never worked on a large enterprise-scale Java project. </Conjecture>

:joy: :rofl: :joy:

You talk about the cognitive load of opening multiple files - I think you need to weigh that against the similar load of remembering “Where is “Form X” in this project?”

Gotcha, I had not thought about folks managing many projects, since I’m only managing one project. Given that situation, predictability across multiple projects makes a lot of sense. Our (only) project features hierarchical data, and our developers using the business language of the organizations we serve is critical. I had thought about adding grouping things together as components makes sense, but now that I’ve read your reply, I’m rethinking the logic around that. Components are starting to feel like sanding against the grain of Django a bit after thinking about your reply a bit.

Part of this also comes from the “one app until it hurts” philosophy.

I see this approach mentioned by developers I respect like you, @adamchainz , and others. In what way(s) does it begin to “hurt”? When following this philosophy, what signs should one look out for that it is time to think about splitting things up? Is it navigating the files, collaboration issues, and/or something else?

We don’t even begin to think about trying to organize into apps until either the models.py file exceeds 2000 lines or we have a clear and unambiguous knowledge that some type of functionality is going to be useful across multiple projects.

I love specific, tactical, rule of thumb advice like this, thank you! I find it very helpful in framing my thinking. The number of lines you mentioned is a lot larger than I would have guessed.

Our experience is that the models more closely interrelate with each other rather than with any specific view.

This makes a lot of sense to me as well. I’d add that you’re likely to reference fields in different models in the same form and views, so keeping models.py in one file sounds helpful.

URLs? Leave them alone.

What about putting the forms and views at the top of urls.py as Carlton suggests? I could see that being helpful. You could start with something like below for a decent size app and split things into separate files, then modules/apps as needed.

project
core (app)
  |
  migrations
  admin.py (it's a little sad we can't put this in models.py)
  apps.py (this is mostly set and forget, so keep it separate)
  models.py (models, managers, and finally signals at the bottom)
  urls.py (forms, views, and urls)

For a large app, we’ve reduced the number of files in our app by about 50% and haven’t given up much that I can see.

5

Thanks for watching the talk. I hope you enjoyed it.

Your talk was wonderful! It made me think more deeply about Django while managing to give actionable advice. That’s the best kind of talk, in my opinion.

I wouldn’t want you to read too much into the points I made there: as I stated/disclaimed several times, the talk concerned what I’d been up to, and the L.o.B. points are more relevant when your code is new and in flux.

Thanks for the restating when you find this sort of locality helpful. My application is in the very early days, so your advice feels very “right” to me right now. Thanks for the link to the The Single Folder Layout for Django Projects, somehow I had missed that!

The links to django-unicorn and tetra are much appreciated. Tetra is certainly something to keep an eye on, but it is still cooking it would seem. I looked at django-unicorn, but at the time it didn’t fit my (weird) brain. I should probably give it another look now that I have a bit more Django seasoning.

If you do go mad, please don’t say that I told you to do so :stuck_out_tongue_winking_eye:

It’s far too late for me to make those sorts of accusations against anyone!

1 Like
6

We have, empirically, decided that around 2000 loc in an individual file is the beginning of the range where we start to think about refactoring / reorganization of the code, trying to ensure it happens before we reach 2500. (And even that’s not an absolute upper limit. In our largest system, we have four files larger than that - not counting files containing tests, data fixtures, or models files representing external reference-only databases.)

In our case it’s almost always only based on file size, and that tends to be based on the usefulness of the tools like code outliners and the “minimaps” that different IDEs and editors provide.

An interesting thought - but as our view files tend toward being the largest files, I’m not sure I’d want to have the urls buried at the bottom of a 2000-line file. Again - it’s an issue of how I use that file. I can cat the file directly, open it in an editor, or look at it in our gitlab instance - and everything I need to see is on the first page.

In addition to what you’ve listed above, in any app we typically have five or more of the following:

management/
static/
templates/
templatetags/
tests/
fixtures/
serializers.py (drf-related or drf-style work)
tasks.py (for celery tasks)
utils.py (general code used in multiple places)

Superficially, it doesn’t seem to me that in our case, saving a file or two here and there makes a real difference. (YMMV)

1 Like
7

Your views are your largest files, that’s very interesting. Right now, my views are about the same size as my models. I guess I had assumed that because of the widely preached practice of “slim views fat models”, that views.py would be smaller than models.py in most projects. I believe combining views and forms still might make sense when they’re still small (when combined), which Carlton mentioned in his talk.

With all of those files in most apps, it seems I need to get better at juggling files, but that comes with time and reps I’m sure. Bouncing between files is already far less disorienting/frustrating than it used to be when I was first learning Django.

Thank you for a window into the world Ken! It has opened my eyes and I have learned quite a lot. I really appreciate the time you spent opening my eyes. Very cool!