Site-specific Middleware and URLs

Using Django Deployment
1

Say I have two sites that I want to serve from the same deployment, but I’d like to have them use different middleware and URLs. For example, one site might be a conventional, server-side template driven site, and another is a REST API. Or another site might have nothing but the Django Admin enabled. Is there a good way to do this?

I realize that a straightforward way to get the same end result is to have separate settings files and deploy them as separate sites that share a lot of code. I’ve done that sort of thing in the past. But these sites share enough code and models where I would prefer to be able to deploy it as one thing and not have to worry about coordinating separate deployments.

I’ve also done things where gating happens upstream, e.g. having nginx configuration for two server URLs which reverse proxy to the same Django for their requests, but allowing requests to /admin to only happen from one of them. That sort of power is great, but I wanted to figure out if I could get a simpler version where I can get a similar effect but push just Django stuff to a PaaS without the nginx layer. I’d like to make it as simple to deploy as possible.

I think I could do the URLs half of this by making custom middleware that swaps out the urlconf attribute on the requests at the top of the chain (as described in this great article by @valentinogagliardi). That’s a little more surprising-to-new-devs than I would like, but I could imagine leaving lots of comments in the settings and urls code and living with the tradeoff.

For the middleware, I’ve only been able to think of terrible, terrible, monkey-patching hacks that will undoubtedly cause more trouble than they’re worth in the long run. For instance, having something that wraps middleware methods and can no-op depending on the the requests urlconf attribute.

I think in my ideal world, I’d be able to declare request/response pipelines based on some top level criteria (domain or top level dir of the URL), and have each one of them define its own URLs and middleware configuration. I have no idea if such a feature is even desirable for Django as a whole though, since this seems like very much an edge case for people.

Anyhow, I’m curious if anyone else has done this. Thank you.

2

You can do a single deployment with two settings files. You set the DJANGO_SETTINGS_MODULE for each of the two runtimes. If you are using something like uwsgi or gunicorn, you create separate wsgi.py files. (And yes, I’ve done this for a different purpose - three installations of the same code under three different urls, using three different databases.)

3

Thanks for the reply. :slight_smile: I take your point that I could spin up different gunicorn instances pointing to different settings/wsgi files and have them run on the same box. I’ve done that in the past as well, and it’s probably the most practical thing to do for this use case. The thing I dislike about that approach is that we end up wasting a lot of memory on over-provisioning some parts (like the admin), without really getting the isolation benefits of putting it on a separate box. We’ve also had instances where the settings drifted in subtle ways that changed model saving behavior. I guess I was hoping that I could serve all requests with the same processes and have a simpler scaling and settings story, but maybe that’s just me being silly.

I wonder if would be possible to have a single WSGI wrapper that mounts the different Django projects as separate WSGI apps and dispatches to them, or if there’s too much implicit global state for that to work correctly.

In any case, I think I’ve wandered out of the realm of practicality and into the realm of “potentially fun hackery that I shouldn’t put in production” at this point.

Thanks again!

4

Your second instance doesn’t need to load the admin, along with not loading the middleware or apps that the second instance doesn’t need.

And, in reality, how much memory in an inactive admin component do you really think is being used?

On the other hand, this:

is a valid point, which could lead one to a “two-tier” settings model - a “common-settings” that is loaded by both of the other “individualized” settings. This is actually the mechanism we use in the single-deployment-three-configurations system I mentioned earlier.

Because of the different global-level registries (models, apps, urls, etc), you might be able to do this - but it’s not going to save you anything. In fact, it’s probably going to make things worse, because this implies that each server process is loading two complete instances, doubling the memory requirements.

Don’t forget that the best configurations of the runtime are going to start multiple processes anyway. There’s no net difference between running eight processes of one configuration or four processes of each of two configurations.

5

Wouldn’t the admin still likely need most apps loaded? (That’s maybe 140-150 apps in the case of the project I’m thinking of.) Granted, it would get a small set of workers relative to the others. I think the bigger case where I could see load be more variable would be things like the REST API. Any sort of pre-fixed ratio of REST API gunicorn workers to regular site workers means that scaling one side potentially wastes a lot of memory of the other if the two don’t coincide (like when your favorite clients start crawling the APIs at off-peak-hours).

And honestly, that’s probably fine. Multiple gunicorn worker types on the same boxes as a simpler, easy to deploy solution that is good for most folks, and if people really care about scaling costs, they run them on separate boxes.

Right, we did this. Still do it, actually. But occasionally things slipped through like…

… like apps that people think are specific to a particular instance, but in fact listen for a signal emitted by something else to update their state based on changes in another app.

Granted, this can be solved to some extent with policy, like blanket rules of “don’t change INSTALLED_APPS outside the common config” and such. Or even custom system checks to make sure those values don’t drift, etc. But having it always run in the same configuration removes some opportunities for foot-guns.

And then each instance inevitably gets its celery settings counterpart…

Agreed, that makes sense.

So if I really did want this use case to work the way that I originally described, I’d want to do it all within the same Django project. And doing so would require either a new routing feature (that is probably not worthwhile for the project to adopt), or else some silly hackery that is more trouble than it’s worth and will leave headaches for future maintainers of the project I’m working on.

I’ll probably try the hacky thing out just for kicks and personal learning, but I’ll stick with running separate sets of workers for the actual solution.

Thanks again @KenWhitesell. I always really appreciate how supportive the community around Django is. Take care.

1 Like