That makes sense, thank you! I see how teams could benefit from the apps namespacing. Would you agree that developers who work as a team on huge project are still in minority? What I’m suggesting is to leave everything as is for teams, but improve things a bit for solo developers or people who are just starting out with Django because they are the majority. Like pre-generating a “core” app when starting a project, which they could split into separate apps as they grow or something like that. You get the best of both worlds. What do you think?
I know exactly what you mean, but this is probably where you and I differ - I think instructing people to design things with multiple apps from the start is the right approach, honestly. I don’t mind if you’ve decided one app is for you, but for those who are unsure, I like suggesting that you design your code as a set of loosely-coupled apps. In my view, it adds a tiny bit of friction up front for a lot of benefit in terms of separation, future scaling, refactor-ability, and so on.
Given that, you’ll be hard pressed to convince me we should change the new-project experience!
3 Likes
I agree with Andrew here.
1 Like
A slightly different take: imagine for a moment that we’re not talking about Django, but about web development in general. We could – and some frameworks encourage doing this, at least at the start – just write a single file and call it the_website.py and put absolutely all the code for our web site in that file.
But after a while that would get unwieldy. If, say, someone needed to go edit a particular thing, or add a new feature, there’d be a lot of code for them to scroll/search through to do it, and even with good code-indexing and searching tools the cost, in developer time, of finding the right things in that file would grow and grow.
So at some point we stop using the single-file approach and instead start maintaining separate files. And probably we settle on a way of doing this that’s based on some kind of logical structure. MVC is one option, but not the only one. And things get better for a while.
But inevitably the code will keep growing. Say we went with MVC; at some point we might have dozens or hundreds or even thousands of data-model classes in the models file, and the same for the other components. The problem is back: how can developers effectively engage with and work on this?
And again the answer is usually going to be separating things based on some kind of logical structure. One possible structure (again, not the only one) is to examine the functionality, and group together the bits of code that implement each general piece of the site’s functionality. So maybe all the auth and user code in one place, all the blog code in another place, all the marketing-lead-form code in another place, and so on.
And at this point we’ve arrived at something resembling Django: our site is composed of multiple pieces of functionality, each of which is implemented in a relatively self-contained way, and each of which uses a standard organizational pattern for its own components (models, views, and so on).
Now, not everybody necessarily needs all this – there will be plenty of projects that never grow all that large – and not necessarily everybody who needs this will do it (I’ve worked on my share of codebases that used the “only a few files, all of them huge” approach). But as a general pattern, it’s useful enough that it might be worth showing it to people up-front and encouraging them to adopt it.
And especially for Django, which wants to provide at least a certain amount of functionality pre-implemented for you, this is a useful approach. Imagine if, in order to write your first Django app, you had to scroll around in files that also contained the complete implementations of the contrib apps, for example. Even for experienced developers that would probably be annoying and difficult; for newer developers it would be much worse. So rather than starting with the “everything all in one place” approach, it makes sense for Django to introduce you to the idea of separating functionality into specific apps right from the start.
5 Likes
Apps always felt too much ceremony for me too so a couple of years ago we went to one app by default and never regretted about that. This is a great article on that topic and it has other useful practices too.
Basically from my experience it’s not clear how apps helps until you want to reuse code between projects. You can easily avoid namespace conflict with some extra levels eg. static/<module>/image.jpg or template/<module>/template.html. To make apps loosely coupled you need to make an extra effort and plan beforehand and this is very hard to accomplish in agile world. I often even don’t know in what module should I place my model and it’s quite hard to move models between apps. In one app setup I don’t fear to make mistake by placing model in wrong module because it takes a couple of minutes to move it into another one when things got clear (and you know the fear of refactoring is the worst thing).
When I see that some module is really can be reused between projects or it feels like it better to be separated from the rest of the project to prevent developers to place project specific logic into it then it can be moved into separate app quite easily because it’s already loosely coupled with the rest of the application that’s why we are moving it from the main app. Even if it contains some project specific entities they can be quickly refactored out of it.
If you’re going to try one app setup then one thing that I would recommend is to get rid of app prefix in table’s names, follow article’s advise and always set db_table explicitly, with that change writing and reading sql become a lot better. In future you may use a checker that warns developers if they added model without db_table so naming tables won’t require extra mental energy.
Overall apps is a useful concept but I would recommend everyone to give a try one app setup and see if you move faster with it and do you face any problems that wouldn’t be possible in multi app setup. Would be good if Django docs had some high level overview of both setups so people knew that app separation is not required thing in Django.
UPDATE:
One thing I want to clear up is that we still use apps like module structure, so if we have one app named mystore inside it we place modules like this:
mystore/
models.py
users/
models.py
views.py
products/
models.py
views.py
this way we have many good properties that @andrewgodwin said apps have, eg. easy to remove some feature if it become obsolete and separation of concern.
4 Likes
This is a topic I’ve been thinking a lot about lately. I’d really like to agree with @andrewgodwin and @ubernostrum here, but I see a problem with that approach: It works as long as the separate apps are, as James and Andrew mentioned, self-contained and loosely-coupled. But as the project grows, things have a tendency to entangle themselves, like a set of cables left in a backpack. I’ve seen this happen especially in teams where there are relatively junior devs and lax discipline. You start with a division to apps that seems logical, but as the requirements change, as in-project utilities are developed, etc, suddenly you start to see dependencies in all directions. App A models depend on app B models, while app B views depend on app A utilities. Foreign keys and M2Ms are added in both directions. And Django supports dynamic and rapid development, this can all be made to work without too much effort.
At that point, refactoring is nearly impossible. Moving models between apps is not supported by Django; it requires custom migration operations, and even with them it’s not trivial. If you’re using the Admin, moving models between apps will change your UI. Even moving views between apps without affecting front-end code is difficult, if you did things the recommended way with separate urls.py files. each included with a prefix. To keep the URLs as they were, urls.py from one app needs to import views from another app, and there goes the loose coupling you’ve been trying to regain.
I am all for the vision that Andrew and James are promoting. We should be teaching people to break things up from the start. But currently, Django makes it very hard to correct mistakes in that division, or adapt it to changing requirements. In my opinion, the set of tools for this kind of refactoring is one of the greatest gaps Django still needs to fill, and we may come to find that we need to change some of our tenets in order to support them.
9 Likes
So am I right to assume that when you placed a model in an app, the models stays there forever (even if it no longer makes sense), unless you’re willing to do manual database work which doesn’t make any sense (that is why we use Django in the first place)? A problem like that would never happen with using a single app per project (you can shuffle models between modules at any point in development).
How is this a great development practice? This approach requires to spec out the project in perfect detail upfront to model the apps and models, in other words a waterfall approach (or live with the fact that your code structure doesn’t make sense as your codebase grows). The Django website says that the framework “encourages rapid development”, but encouraging the use of apps basically prevents to use agile practices effectively.
3 Likes
I think you’re overstating the case a little for how hard it is to move models around 
It’s totally possible to develop iteratively while splitting your code into multiple apps! It’s not particularly pleasant but you can do it using just the migrations framework. I think it would be great if Django was better at handling this (but I’m not volunteering to do the work so I accept the compromise). I’ve done this kind of surgery on a work project a few times and it’s not been a big obstacle.
Sure, nothing will save you from bad process. But having your models divided up into different apps can avoid unnecessary process when different people are working on different bits of code.
Could you please tell me more about the process of moving a model from one app to another (considering this model has foreign keys with models from other apps)? I assume that this is why Django docs recommends to start a project with a custom user model since it’s a nightmare to migrate the model later
Could you please tell me more about the process of moving a model from one app to another (considering this model has foreign keys with models from other apps)?
Can’t you just set db_table to keep the old table name ? Of course you will need to change the app name in your foreignkey definitions but then I think it should work.
Exactly what I did in this situation, plus for deployment I had to delete some lines from django_migrations and fake apply them. Not really sure of the top of my head, but let us know if you would like more detailed a tutorial.
It’s more complicated than that.
Literally the first line in the answer to the question “how to move a model between two apps” is “*don’t do it!!*”.
1 Like
I don’t see how the other way presented in the stackoverflow answer invalidates my suggestion, but anyway the answer proposes pretty much a custom migration in addition to code changes which seems like a routine operation to me. But like someone in the ticket said IIRC: perhaps Django could provide a specific operation for this kind of migration to make it easier.
Also don’t miss that part:
Would you like to suggest ways Django could make it easier ? Perhaps not removing apps, I recon we’re probably many to love the app system, not only to ensure code isolation but also to ease reuse.
I’m not suggesting to remove apps. I’m suggesting to use single app per project.
The only real feature of using multiple apps per project approach is that you can have two models with the same name within a project (by defining them under separate apps).
Another potential feature is that multiple app reduce migration conflicts. I don’t fully agree with this one since apps don’t solve this issue per se, development process does. Unless you define the development process as “each team member should only work on a single app at a time”, nothing stops a developer to work on the same app as their coworker. So if we’re defining the development process we might as well specify how the migrations should be done when doing the work on parallel branches. Here’s a thread about this topic. Finally, if a team experiences migration conflicts and decides to solve it by assigning a team member per app, that means that the app structure is now driven by a team-app assignment (new apps might be created so that devs could work without conflicts) rather than organically (moving code to a separate app so that the code structure makes more sense). What I’m trying to say is that apps are not a solution to the migration conflicts problem.
Now when it comes to disadvantages we have the problem of moving models between apps. Saying that it is trivially fixable by writing migration files manually is like saying: “if you have a migration conflict, you could just solve it by manually modifying the migration files”. The point of a high level framework is to speed up the development process as much as possible. Another problem is that developers are confused about the difference between apps and projects. It is also not clear how to structure your code when you start a project: Django forces you to come up with a name for the project and a name for the first app. This assumes that we know the total number of apps (and the models) we’re going to have upfront – this is not agile.
Sure, but some things are more common than others: especially in smaller teams, it’s common that people work on the different bits of the code - and if you’re using multiple apps, this avoids migration conflicts. I have a team of 3 and conflicts have happened a couple of times in the last year with multiple (~11) apps. If we had one app it would be something we’d need to deal with every week. Moving models around, though, happens fairly rarely. I think I’ve moved around 3 or 4 models over the last year. Something doesn’t have to be perfect for it to be good or useful
1 Like
Apps are one of the best things in Django. And I miss them so much when I have to work with other frameworks!
Apps let you split your code, keep it more organized, allow multiple teams to work on different parts of the project.
I see apps as Python modules on steroids; the only thing that differs an “app” from a package is the presence of apps.py file with a config class in it. That’s all; if you don’t use models or templates, you don’t have to follow conventions. Views can be put into controllers.py if you want to, there aren’t many limitations.
But to keep your code more organized, it’s an excellent habit to follow well-established conventions. Trust me. You’re not a single person who’ll be working on the project, think about yourself coming back to the codebase in a year, that will be another guy, not you, help him today!
And regarding early buy-in, this is total BS; you can refactor your code anytime. Split it into even more apps or merge several apps. The only thing that could get in a way is the table name of the model. By default, its name constructed from the app name and model name, but if you manually set table names in a model’s options, the issue will be gone, just get used to it, do something like:
class Page(models.Model):
class Meta:
db_table = 'page'
I hope it helps.
1 Like
TBH, you get a sense of what app you need to make to host what model and in which directions you want your apps to be coupled after using Django for a while.
I think the easy way would be, before adding a model: ask yourself why it shouldn’t be in its own app, this should save you some model moving in the future.
However, I do like the idea of grouping a model with its features such as views, urls etc … wait, that’s exactly what apps allow you to do.
Maybe check how django-autocomplete-light is structured: it’s a python package which is composed of several python modules, that’s also something setuptools allows you to do.
Just as an anecdotal data point: I don’t work in large teams/multi-team projects, I usually work on my own or with up to two or three others. I still enjoy apps a lot, even if I may not always use them as intended. When my projects have strongly separated frontend parts (like an administrative one, one for the API, and one for customers/users), I like to separate them into different apps for URL and template namespacing. This breaks up huge template/view modules into smaller ones, which helps me reason about the structure of my application.
(I have less use cases for splitting models over apps – I usually start out doing it, and then later it turns out that my models are more tightly coupled than I thought in the beginning, and that shoving them in a core or default app would’ve been better, or at least no worse.)
This is indeed “use not as intended” – your apps are not defined by the functionality they provide, but rather by the way the functionality is exposed. It could be said that where the recommended division (and the one most fitting reusable apps) is “vertical”, yours is “horizontal”.
That is interesting.
And regarding early buy-in, this is total BS; you can refactor your code anytime. Split it into even more apps or merge several apps. The only thing that could get in a way is the table name of the model.
Table name is not the only problem, from the top of my mind I can say that content type framework may cause problems during model movement.
I have a team of 3 and conflicts have happened a couple of times in the last year with multiple (~11) apps. If we had one app it would be something we’d need to deal with every week.
That’s true but it doesn’t feel like a big problem, 99% of such conflicts can be easily fixed with rm migration && makemigrations or just makemigrations --merge, on your CI you can have makemigrations --check step that will fail if pr has migration conflict.
Hi. I am fairly new to Django, although not completely alien to web development (Rails, Phoenix,…).
My understanding of “apps” so far is it’s just a way of grouping related functionality. Sort of like putting stuff naturally to folders. So you’d have everything related to users in one “app” (folder), and everything related to products in another “app” (folder). Each of these apps could be a few models, but it shouldn’t be a single model file with 20 models - that’s probably a good sign stuff should be split.
Also another guiding principle I’m thinking of is that app is typically something that lives under its own URL endpoint. So for example /users (and all related stuff) and /products (and all related stuff). Although I understand this is very rough estimation, it’s never really simple as that.
Rails has fairly different way of organising stuff, but Phoenix does have something called Contexts, which seems fairly similar to Django apps. See Contexts — Phoenix v1.5.7
Would you say my understanding is ok-ish, or am I completely wrong and misguided somewhere, please?
Thank you!
1 Like