I have 5 APIs (and growing) that regularly download (currently outside Django, but will be migrating in) into seperate database tables, roughly one per API with relationships. Machine sensor data (x2), weather, geological, geocoding + whatever else comes up in the future. The relationships are mostly based on per machine and per sensor type, with the “master” being per machine. Also within those API apps, there are a few functions that process the raw sensor data (a lot of it, and messy) into other tables, e.g. cleaned, first/last/average values for fast user display on dashboards per machine.
Raw data is used for stat analysis and ML prediction models.
My thinking at the moment is to create a polling app which would host (not sure of the right term) those tables (models). These apps would manage the data in those tables. Apart from users, there is little or no user Create, Update or Delete interaction with the database data. Mostly Read.
I would then create other apps that would bring together all these various tables for display to different types of users in dashboards. However this would mean importing the polling app models into the other apps, and everything I have read says that all apps should be self contained. Also there would be relationships across apps which makes me shudder. Again all should be coming from the machine id, not the other way round so trying to avoid circular dependencies.
The other apps will be things like machine summary, fleet summary, aftermarket sales and technical support dashboards which will use the same models (albeit with slightly different queries), but feel like different apps. There will be some new models for the different apps, however they will share the same core processed API data.
For various reasons, I already have a proof of concept built in MATLAB with the database tables up and running and works…as a PoC. My thinking of the structure probably comes from creating that.
I am now learning Django for the production version, however I am struggling to see how to fit within the principles of Django I have picked up:
- Keep apps small
- Each app does one thing well
- Keep apps separate so they can be exported and used elsewhere
Is this structure normal/sensible for Django, and apart from creating one massive app, should I be rethinking the approach? Help!