Why do we need apps?

Django Internals
1

What is the use case? Why differentiate between a project and an app?

A common explanation is to structure code within a project or to be able to ship an app as a standalone package. However, I’m still not sure what benefits the first use case provides. Finally, why would we need a Python package to be an app in order to be able to work with Django? I guess my main question is why can’t we just use regular Python modules to structure code? The only API that apps provide is the ability to dynamically load app models and configuring the display name of the app in Django admin. Can’t we do the same with regular import or any other less complicated mechanism?

My main problem with apps is that while it doesn’t provide any tangible benefit, it introduces a lot of confusion for those who just start learning Django and a lot of unnecessary complexity in other parts: like staticfiles, templates, etc. Am I missing something?

9 Likes
2

The main purpose of apps is, in my eyes, to provide logical separation of reusable components - specifically, a first-class namespace for models/admin/etc. - and to provide an easy way to turn things “on” or “off”.

In some ways, it’s a relic of the time when Django was created - when Python packaging and modules were much less developed and you basically had to have your own solution to the problem. That said, it’s still a core part of Django’s mental model, and I think INSTALLED_APPS is still a cleaner, easier solution than Python’s replacement offering of entrypoints (which makes it quite hard to disable a package that is installed in an environment but which you don’t want to use).

Is there anything specifically you think could be decoupled from the app concept today? Models and admin need it for autodiscovery and a unique namespace prefix, so that’s hard to undo, and I’m struggling to think of other features you need it for (in fact, if all you want is just a library, you can make it a normal Python one - no need for the app wrapping unless you’re shipping models, templates or admin code IIRC)

3 Likes
3

Thank you for the explanation, @andrewgodwin!

I mean I’m not sure why I, as a user, should care about apps if my goal is to build a web site that I’m not going to distribute or anything (this is the case for the majority of Django developers). In this use case I would create a project and an app (because I need to define models, etc). But why would I create any additional apps in this project if I’m not going to distribute any of them? Any logical separation can be achieved via Python modules. I’d say using a single (locally created) app per project is more durable since there are no templates, staticfiles or any other namespace conflicts. Also no cyclic imports.

Apps force you to structure your code too early. When I’m starting a new project I usually create an app with the similar name as the project since I have no idea at this point how my finished project will look like exactly and what parts should go where. This StackOverflow question sums up this issue perfectly.

Maybe I’m overlooking some use cases, but if my goal is to create a website that I’m not going to share, would I miss something if I just stick to a single app per my project? If that’s fine to use a single app per project, then can we change something to simplify the job for new developers? A lot of newcomers (and people who already work with Django, like me :slight_smile:) are confused about this concept. A couple of ideas:

  • Change documentation to advise people to not create more than one app if they don’t plan to distribute them. Make it clear that the use case for extra apps is the ability to distribute them
  • When starting a project, create an app for them automatically. First, they can’t work with models or other Django stuff without an app, so they would still have to create it. Second, the name doesn’t matter much since this would likely be the only user-created app in a project. Finally this will save some keystrokes: we can automatically add this app into INSTALLED_APPS, automatically include urls from this app in project’s urls.py, create templates or static folder, etc
  • Not sure if it’s possible but maybe merging a project and an app somehow (?). This is basically how it’s done in other frameworks I worked with (Rails, Yii)
  • Perhaps changing terminology would help (?)

What do you think?

3 Likes
4

Sorry, this post is a bit long! But apps are useful for splitting code up, not just for distribution.

At my work we have a Django project which currently has 11 apps, with 35 models split over them. It would be a lot messier if all the models were defined in just one place, because the apps that have different functions. For example, we have an app for a membership database, a smart meter data viewer, and one for sending notifications to users. They all do different things and they each own the data models they need to function. There are obvious, if fuzzy, boundaries between the different bits.

It’s an advantage when, for example, one person is working on one module, and someone else is working on another. You can both change the data models independently and make your migrations without conflicting with each other. If we had all the models together in one app it would be a lot more conflictual.

That said, someone at my work was recently confused about what the difference between a Django app and a Python module is - so I understand that this can be a problem. But a Django app is “just” a Python module which can have its own models. I think it’s useful for people to understand how to make use of this, rather than to paper over it.

I also agree that it’s easier to have everything in one app at first because you don’t know how things are going to be best split up. But in my experience you’re going to want to split it up at some point, and you might as well learn to take advantage of the way Django lets you do that!

All that said, an engineer at Octopus Energy recently published quite an unorthodox approach to structuring a Django project that does in fact minimise the number of apps and keeps all the data models in one module. So that might be interesting to you too - but it’s a big departure from almost everything else I’ve seen about how people use Django.

4 Likes
5

I mean, I often write sites using just one app called core in the model you’re describing. The app concept is much more useful for agencies or those maintaining a set of sites with similar functionality, where you absolutely do want to have parts be re-useable.

That said, it is generally cleaner, model-wise, to have things spread across separate apps - for example, my most recent project could have just been a single app, but instead I did three just to make the template and model separation clearer.

Given that Django’s philosophy is that you can turn parts of it on and off, and those are presented as apps, I’m not sure I want to really backpedal much on the app concept. It’s nice to show new developers that the admin, or sessions, are not special and are just apps themselves - and they can be removed or, crucially, replaced. Many frameworks don’t do super well at the “just replace parts of it as you outgrow it” part, and I feel the app model (and the changes we have done over the years, such as dynamic foreign keys) plays into that.

Apps are basically just Python modules with extra bits, in the end, and the extra bits are basically all autodiscovery. I believe you could write a Django project entirely without user-created apps, but you’d have to do all the model discovery and registration yourself.

6

Isn’t it possible to achieve the same with regular Python modules? We could create a models directory and create a subdirectory for each component, same goes for views, etc. However, this approach allows for code structure to grow organically rather than forcing to split everything into apps from the beginning.

It seems that it’s more about the development process rather than something inherent to apps. Multiple people could still work on a single app and they would still run into migration conflicts if the process is not organized properly.

1 Like
7

This is the use case for people who are going to distribute their apps in future and using apps in such scenarios makes perfect sense.

Nothing stops you from separating the models or templates by creating subfolders inside models or templates folders. This is essentially the same thing except we’re not introducing any new concepts or force users to adopt a certain approach in the beginning.

I’m not sure if it’s completely true for user-created apps: if I split my code between multiple apps, the code would still import stuff from each other and removing the app from INSTALLED_APPS wouldn’t be enough. My idea is simple: apps concept makes perfect sense as a tool for code you need to distribute, but I can’t see any value from applying this approach for a regular project.

1 Like
8

In sort, I think it comes down to - yes, you could use Python modules, but then what do you use as the namespace for templates, models, migrations, and so on?

The full module path is ugly and not stable enough to use in a database table. The “last” path of the path is ill-defined if you put some things in a submodule. And to add on top of that, apps can be installed twice with different aliases and paths, or have their python module moved around as part of a code cleanup but have their alias kept the same.

Apps are basically just a discovery plus alias mechanism for Python modules - a necessary one, given Django’s design. I am unconvinced that using plain Python modules in their place, and the increase in code and increase in mysterious errors that result, is a user-friendly action. Maybe it’s cleaner, in some sense, but I feel that it is less pragmatic.

1 Like
9

What do you mean? If we use a single app for the whole project then you have this namespace, no?

Could you give a simple example please? I’m not sure I understand.

10

Indeed, but at some point, you are going to want to use something from someone else (unless you truly want to write everything in house), and at that point, you need the namespacing. Imagine if you used a third-party app for social login but it collided with your existing models or templates because there was no way to namespace them.

Let’s say I have a Python module called rfid_inventory.directory. What should we use as the prefix for models created from this module - say, an Item model? (See above why you need prefixing as a single developer, and of course if you have multiple developers, it’s even more important to save your developers from having to check every existing model name before they make a new one)

  • If we use the full path, rfid_inventory.directory, we’re going to get a pretty ugly table name like rfid_inventory__directory__item. That’s maybe doable, but…
  • Your model is probably actually in the module in rfid_inventory.directory.models.item, making the table name rfid_inventory__directory__models__item__item. That’s not a good schema at that point.
  • If I rename my app’s source code directory from directory to items, now I have to rename all my tables in production in order to ship my app. This can’t be done without downtime.

The reasonable solution is to say “let’s put a prefix attribute on the Meta of each model to stop this”, and lo and behold, that’s what apps do for you by default.

I still think you’re correct that a small, single-developer project probably doesn’t need apps, but Django is generally about trying to give you a good foundation that’ll scale to more developers and more traffic. If we took out everything that small, single-developer sites didn’t need, it would be a framework that basically never got any traction.

Besides, Python already has a great app-free, small, single-developer-friendly framework - Flask. I think Django’s App abstraction is a very reasonable tradeoff to make your project much more expandable in future - without that small amount of upfront effort, which we almost entirely template away, you would run into a wall as soon as you wanted to start using third-party code.

1 Like
11

Thanks for the example! I think you probably misunderstood my suggestion. I’m not saying to ditch all apps infrastructure. I’m saying that we should work with a single user-created app per project. It would still provide the namespacing, would keep backwards compatibility with other parts. Most importantly, it would be less confusing for developers who create new projects, it would allow the code structure to grow organically.

1 Like
12

I mean, again, that is great for single-developer projects, and you should feel free to do that! But projects with tens of engineers benefit massively from splitting into apps to deconflict all their changes and source code, so I don’t think we should make Django pop out a “default” app, nor do I think it’s good advice for most people who have aspirations of a site that grows even moderately larger.

1 Like
13

That’s what I’m trying to understand. What benefits do apps introduce?

You said:

  • deconflicting changes and source code: apps intrinsically don’t solve code conflicts, this is about development process. Even if you split your project into multiple apps, nothing stops multiple people to work simultaneously on a single app. Expecting each team member to work on their “own app” won’t solve conflicts unless you create a robust development process which documents how changes should be introduced. This is a “people problem” not a software problem.
  • namespaces: this is only relevant when you use 3rd party apps, and single app solves this problem. Locally, everything except models can be namespaced naturally with directories or Python modules. Finally, you wouldn’t use two models with the same name in a single project. This sounds like a data model problem or like a situation when you do need to use a new app. But how often do you experience such problem? If it’s a niche use case then it definitely shouldn’t be a default approach.

Have I missed something here? I’d be happy to learn more usecases of apps. I’m not suggesting anything radically new here. I’m not suggesting to break backwards compatibility or anything. I’m trying to suggest an improvement that would simplify the mental model for the majority of developers, while retaining the ability to create more than one app per project for people who really need this (those who distribute apps or teams that work on huge projects). As I suggested earlier we could approach it from either documentation or project scaffolding perspective. All major frameworks work that way (Rails, Laravel, Yii etc) and developers build massive projects using these technologies. I think it’s valuable to developers to use technology that introduces the least amount of surprise.

14

Namespacing is super important when it specifically comes to the ORM, and specifically migrations. Allowing teams to develop apps one-per-team and coordinate the migrations among themselves is a lot easier than trying to keep a single global migration history.

Also:

I’ve seen this at least three times, so it absolutely happens. You’re right that you can solve a lot of these problems with coordination, but most large development teams are bad at that, and so having a natural way to separate apps from each other and reduce the communication complexity that teams need is welcomed.

15

That makes sense, thank you! I see how teams could benefit from the apps namespacing. Would you agree that developers who work as a team on huge project are still in minority? What I’m suggesting is to leave everything as is for teams, but improve things a bit for solo developers or people who are just starting out with Django because they are the majority. Like pre-generating a “core” app when starting a project, which they could split into separate apps as they grow or something like that. You get the best of both worlds. What do you think?

16

I know exactly what you mean, but this is probably where you and I differ - I think instructing people to design things with multiple apps from the start is the right approach, honestly. I don’t mind if you’ve decided one app is for you, but for those who are unsure, I like suggesting that you design your code as a set of loosely-coupled apps. In my view, it adds a tiny bit of friction up front for a lot of benefit in terms of separation, future scaling, refactor-ability, and so on.

Given that, you’ll be hard pressed to convince me we should change the new-project experience!

4 Likes
17

I agree with Andrew here.

1 Like
18

A slightly different take: imagine for a moment that we’re not talking about Django, but about web development in general. We could – and some frameworks encourage doing this, at least at the start – just write a single file and call it the_website.py and put absolutely all the code for our web site in that file.

But after a while that would get unwieldy. If, say, someone needed to go edit a particular thing, or add a new feature, there’d be a lot of code for them to scroll/search through to do it, and even with good code-indexing and searching tools the cost, in developer time, of finding the right things in that file would grow and grow.

So at some point we stop using the single-file approach and instead start maintaining separate files. And probably we settle on a way of doing this that’s based on some kind of logical structure. MVC is one option, but not the only one. And things get better for a while.

But inevitably the code will keep growing. Say we went with MVC; at some point we might have dozens or hundreds or even thousands of data-model classes in the models file, and the same for the other components. The problem is back: how can developers effectively engage with and work on this?

And again the answer is usually going to be separating things based on some kind of logical structure. One possible structure (again, not the only one) is to examine the functionality, and group together the bits of code that implement each general piece of the site’s functionality. So maybe all the auth and user code in one place, all the blog code in another place, all the marketing-lead-form code in another place, and so on.

And at this point we’ve arrived at something resembling Django: our site is composed of multiple pieces of functionality, each of which is implemented in a relatively self-contained way, and each of which uses a standard organizational pattern for its own components (models, views, and so on).

Now, not everybody necessarily needs all this – there will be plenty of projects that never grow all that large – and not necessarily everybody who needs this will do it (I’ve worked on my share of codebases that used the “only a few files, all of them huge” approach). But as a general pattern, it’s useful enough that it might be worth showing it to people up-front and encouraging them to adopt it.

And especially for Django, which wants to provide at least a certain amount of functionality pre-implemented for you, this is a useful approach. Imagine if, in order to write your first Django app, you had to scroll around in files that also contained the complete implementations of the contrib apps, for example. Even for experienced developers that would probably be annoying and difficult; for newer developers it would be much worse. So rather than starting with the “everything all in one place” approach, it makes sense for Django to introduce you to the idea of separating functionality into specific apps right from the start.

5 Likes
19

Apps always felt too much ceremony for me too so a couple of years ago we went to one app by default and never regretted about that. This is a great article on that topic and it has other useful practices too.

Basically from my experience it’s not clear how apps helps until you want to reuse code between projects. You can easily avoid namespace conflict with some extra levels eg. static/<module>/image.jpg or template/<module>/template.html. To make apps loosely coupled you need to make an extra effort and plan beforehand and this is very hard to accomplish in agile world. I often even don’t know in what module should I place my model and it’s quite hard to move models between apps. In one app setup I don’t fear to make mistake by placing model in wrong module because it takes a couple of minutes to move it into another one when things got clear (and you know the fear of refactoring is the worst thing).

When I see that some module is really can be reused between projects or it feels like it better to be separated from the rest of the project to prevent developers to place project specific logic into it then it can be moved into separate app quite easily because it’s already loosely coupled with the rest of the application that’s why we are moving it from the main app. Even if it contains some project specific entities they can be quickly refactored out of it.

If you’re going to try one app setup then one thing that I would recommend is to get rid of app prefix in table’s names, follow article’s advise and always set db_table explicitly, with that change writing and reading sql become a lot better. In future you may use a checker that warns developers if they added model without db_table so naming tables won’t require extra mental energy.

Overall apps is a useful concept but I would recommend everyone to give a try one app setup and see if you move faster with it and do you face any problems that wouldn’t be possible in multi app setup. Would be good if Django docs had some high level overview of both setups so people knew that app separation is not required thing in Django.

UPDATE:

One thing I want to clear up is that we still use apps like module structure, so if we have one app named mystore inside it we place modules like this:

mystore/
  models.py
  users/
    models.py
    views.py
  products/
    models.py
    views.py

this way we have many good properties that @andrewgodwin said apps have, eg. easy to remove some feature if it become obsolete and separation of concern.

4 Likes
20

This is a topic I’ve been thinking a lot about lately. I’d really like to agree with @andrewgodwin and @ubernostrum here, but I see a problem with that approach: It works as long as the separate apps are, as James and Andrew mentioned, self-contained and loosely-coupled. But as the project grows, things have a tendency to entangle themselves, like a set of cables left in a backpack. I’ve seen this happen especially in teams where there are relatively junior devs and lax discipline. You start with a division to apps that seems logical, but as the requirements change, as in-project utilities are developed, etc, suddenly you start to see dependencies in all directions. App A models depend on app B models, while app B views depend on app A utilities. Foreign keys and M2Ms are added in both directions. And Django supports dynamic and rapid development, this can all be made to work without too much effort.

At that point, refactoring is nearly impossible. Moving models between apps is not supported by Django; it requires custom migration operations, and even with them it’s not trivial. If you’re using the Admin, moving models between apps will change your UI. Even moving views between apps without affecting front-end code is difficult, if you did things the recommended way with separate urls.py files. each included with a prefix. To keep the URLs as they were, urls.py from one app needs to import views from another app, and there goes the loose coupling you’ve been trying to regain.

I am all for the vision that Andrew and James are promoting. We should be teaching people to break things up from the start. But currently, Django makes it very hard to correct mistakes in that division, or adapt it to changing requirements. In my opinion, the set of tools for this kind of refactoring is one of the greatest gaps Django still needs to fill, and we may come to find that we need to change some of our tenets in order to support them.

9 Likes