I’m debating how I should structure external API integrations in my Django app for a project that will involve multiple external integrations like the Fitbit API. As I’m mainly experienced with regular CRUD apps and not so much with these types of integrations I would like to understand how you and others structure these in your project.
So far I’m thinking about creating an “integrations” app in my project which will contain sub-directories like fitbit and more with their respective files including models, clients, and tasks for which I am currently using Celery to offload long-running processes.
So my question is, how can I create a structure to integrate these external services without creating too many seperate high-level apps and if there are any best practices for this?
Thanks in advance and I’m looking forward to what you all have to share!
I’m not sure I’m following what you’re talking about in terms of “integrations” here.
Are you saying that your Django project is going to be callng these external services, and that you want to add the data retrieved by them to your system?
If so, my decision would be based upon the degree of interrelationships of that data with the rest of my data.
<opinion>
I’m primarily in the camp of “One app until otherwise proven necessary.”
There are two cases that would most likely cause me to look toward a multi-app environment.
The identification of a module that I could see using in different projects. In other words, if I knew that I was going to use the “Fitbit API Integration” in projects other than what I was currently working on, then I’d consider making it an app. (But if I didn’t have an identified need - even if I thought that I might use it elsewhere, I probably wouldn’t bother.)
My model and view files were getting too large to be manageable. (We don’t start thinking about reorganizing until the files get to be about 2000 lines, and don’t start considering it urgent until we get to about 2500. But those are guidelines and not hard rules.)
In pretty much all other cases, we would create this all as one app.
My Django project will indeed call these external services but the common data that will be used throughout other parts of the application will be extracted into a standardized model for these integrations (Fitbit, Garmin). I normally would also extract them into seperate apps, but as the rest of my project will not interface with the inner details of these integrations I was wondering if I could put these into integrations/fitbit and integrations/garmin.
Again, I’m firmly in the “One app unless proven otherwise” camp.
Unless the project I am working on clearly fits into one of the two categories above, all systems I build start as one app.
It is very rare that I start a project thinking I’m going to need multiple apps. (I think it has happened twice - there might be a third - in 10 years.)
I do not believe that arbitrarily adding apps provides benefits to most projects.
I made the mistake of putting too many models in some of those apps though, which was problematic. Some made sense, because they were super local. But if the model is used from outside the app, then it’s better to consider it general business logic modelling.
Now, you can argue that “app” is silly, but “module” is more sane, and I think that’s probably a good argument
I think I misunderstood your original comment, are you saying that you structure your Django projects completely within a singular app? I.e. keeping models for users, authentication, and your other models in 1 file?
Makes sense! I was also thinking about this as the integrations are more module than app, however the auth/user related stuff for the integration should go somewhere so I’m still stuck with models in the integrations.
I think your point on the general business logic modelling also makes sense and I’ll decouple the more general data/logic into seperate apps which only think about their business logic (i.e. not the integrations).
That’s very interesting, may I ask what kind of projects you take this approach for? More so in a regular CRUD application for which the frontend will do heavy lifting or in general?
I would think that keeping things in 1 singular app will get hard to understand quickly, especially for people that get into the project at a later point.
We find it actually reduces confusion because people don’t need to track what files or directories something may be in. There’s also no additional decision points needing to be made regarding where something will go.
Overall, this approach has proven to be a net-benefit for people bouncing between projects.
