Largish project app structure

Using Django
1

G’day all,

My project has grown considerably in the last six months and I’m currently working through a major uplift to tidy up the codebase before first semester start after summer (Northern Hemisphere).

Almost everything in my app relates to a case. That is, a case has physicals, it has diagnostics, it has treatments and it has many other things.

Additionally, a user performs actions and these actions are recorded. For example, a user chooses a treatment and a treatment has a relationship to a case, therefore the users actions are related to cases. Everywhere I look, relationships always end up back with the case. And to me, this is logical, we’re modelling a case, e.g. The case of the bunny rabbit with a sore foot who was treated by Jenny.

As my code base has grown, I’ve tried to keep things tidy by splitting things out into different apps. For example, the diagnostics part of a case is sizeable in terms of models, views and utilities. I like this, as it keeps thing quite clean and my files lengths are kept to a manageable level.

But as the project grows, I see some downsides to this approach. My URLs are getting a bit too fiddly for my liking and would be much simpler if everything that was related to a case was in the cases app.

So with that in mind, I’ve been thinking about different project structures for my app.

A Single App with single files I don’t like the thought of this one. My models file would be huge as would my views file etc etc etc. The only positives are simple URL patterns and that I can now reuse this app in another Django project. I have no intention of going down this route.

A Single App with multiple views, model, etc files In some of my apps I have a module for serializers and tests instead of single app level files. I like this level of abstractions as my file names can be very concise - nested_serializers.py , censored_serializers.py and admin_serializers.py as some examples. This keeps line lengths to a maneable level, especially with test files.

The drawback here would be that my modules will contain a lot of files. A lot! The upside is that I again have a reusable app, simpler URLs and everything that is related to a case is neatly packed up in the cases app. It’s the “a lot!” of files which deters me from this path.

A Single App with Sub Apps This is an idea which occurred to me recently but I have no experience with it and not even sure if it is the done thing or recommended.

My thinking is that I would have an app, cases and in cases I would have sub-apps for all the major components, i.e. diagnostics , treatments etc.

By doing this I would gain a reusable cases app, simpler URLs than what I have today and a less chaotic file structure than the two structures already mentioned. I’m not sure if there are any downsides to this which is ringing alarm bells, and hence why I’m writing this post when I instead should be enjoying lunch and a cup of tea.

Keep Things as they Are

Of course, I can keep things as they are, but I wouldn’t be writing this if there weren’t some nagging thoughts in my head. At this point in time, this is OK. Little refactoring but I have this gnawing feeling that something just isn’t quite right. I’ve read in Two Scoops of Django that the authors recommend keeping model files small and manageable. I’ve also read that one should keep all of their models which relate to each other in the same app. Both of these make sense to me, but both these things cannot be true in my project. I also appreciated that no two Django projects are necessarily the same, and with that in mind I’m trying to find what works best for my project and not necessarily trying to follow rules and recommendations to the letter.

API on the side My project is for better or worse an SPA. I use DRF and think it is mighty. Currently, all my apps have their API urls in their respective apps. I quite like this as opposed to what I have seen others do, which is to have a separate app for their API. This app typically has no models and contains just the API side of things, e.g. serializers, views, and URLs. Doing it this way seems to be much the same as building a largish Django project with just one app. Eventually the API app will fill up with many, many lines of code and start to be a bit cumbersome to work with - at least that’s what I’m thinking

So with the above said, I think I’m going to stick with my approach of including the API URLs inside each respective app. To me, this makes a lot of sense because the absolute vast majority of my URLs are DRF URLs and the apps are 99% DRF apps.

Recommendations So I’m wondering how others have tackled the issues that I am facing now. What have you done with your apps when you’ve had 50+ models all related back to a single model? Have you gone down any of the routes I have mentioned, or have you chosen another path? I’d be very interested to read what you have done.

As always, thank you for taking the time to lend a helping hand.

C

2

I would do this but turn your modules into packages, that is split them into submodules. You can do e.g.:

serializers/
    __init__.py
    admin.py
    nested.py
    censored.py

The same with models, views, etc.

Using more than one app can help but if there are many relations and they are chagning, it’s often hard to avoid them all depending on each other.

3

No recommendations from me, but I will share some of what we’ve done.

Our largish project has 123 models, primarily relating to 2 key models. We have two large apps (one built around each key model). The largest models.py file has 45 models in it, the second largest has 28.
(This is also a project pulling data from three foreign data sources - each with 133, 92, and 81 models respectively. As a foreign data source, each resides in its own models.py file. There’s no attempt to break them down into any smaller organization, as we see no value in doing that.)

The other apps are functional divisions that span multiple categories. For instance, we have one app that just manages the navigation menu on the page required by our rather convoluted security model. We have another app for generating and managing reports. (We use jQuery datatables for the presentation of reports - selected as much for its ability to export reports to multiple formats as anything else.)

We have three views.py files between 2000 and 3000 lines. Other than the foreign data sources, our models.py files top out at about 1500 lines. (The foreign sources range from 1300 - 2200 lines in size.) I have heard people comment that that’s way too large to be manageable, but it doesn’t seem to have an adverse effect on us. (There’s only 3 of us working on this project, so it’s not like we’re going to be stepping on each-others toes when it comes to merging updates.)

No SPA / DRF. What JSON we generate is pretty much limited to the data being rendered by datatables, and so it’s just a JsonResponse in our views. Everything else is straight HTML with a little bit of JavaScript / jQuery mixed in for some dynamic formsets.

We don’t really pay attention to what the URLs look like. Pretty much every reference in code is generated via reverse. Some do look a little odd, but for this app, they’re not really for human consumption anyway. They can be bookmarked and that’s about all that matters.

So if I were to draw a conclusion from all this, I’d say that “small” and “manageable” is in the eye of the beholder. (I love Two Scoops - to the point that I’ve given hard copies to my teammates. But we use it as guidelines rather than requirements. I think I’ve commented elsewhere on here that their suggestions aren’t absolutes - but create a topic of conversation. It’s better to consciously do something different than to just default to a decision because of not being aware of an issue. That comes from my “Enterprise Architect” training sooo long ago.)

4

Cheers Adam, Ken,

I should have said packages and modules and not just modules when I wrote

Having corrected my lack of clarity, we’re on the same page about creating packages to neatly encapsulate functionality.

Given that I have found myself in circular importer now and again, the thought of having all related in a models file is tempting. If I do go this route, I’ll be at about 3000 lines for my models.py. I have a fair bit of model logic for some of the more complicated functionality so that takes a up line or two. With a bit of planning I should be able to put the models with most relationships in one file and use additional model files where appropriate. That will save some headaches in the future, and as Adam touched on, there will be changes to our modelling. Once this pandemic ends, I’ll politely ask my wife to have my high resolution monitor back. That’ll surely help.

I like the fact that the two of you have recommended/are doing one of my potential designs; that eases my concerns considerably. With a little bit more thought, I see quite a few flaws with the sub-app approach and it wouldn’t have changed anything about how the models are currently imported today. That one is out the window.

I’d better get on it with, I’ve got some copy and pasting to do.

Thank you both.

C

5

A nice document on an opinionated approach to coding style, architecture, and design patterns for a Django app is OctoEnergy’s conventions docs. Obviously that’s for a single company, but it might help you discover some alternative ideas on what you do and don’t like.

In the end for me the quote they lead in with is key:

“Everyone is happier when the code we interact with is clean and consistent. Even if it doesn’t match your own style perfectly, consistency is the most important thing.” Chris Coyier

I do not work for OctoEnergy or am in any way affiliated with them. I found their conventions docs when I was checking out their a job postings.