How to route domain names to a specific view?

Using Django
1

(Important: this is Django 1.8)

In our site we are going to introduce a feature that would allow users who own a domain name of their own to link it to their organization id in our site. That way they can bootstrap their organization site through our tools and link different relevant pages to their domain. We’d provide default content so they have an organization page based on their organization information we already have in our site right off the bat.

Basically, using an example, if an organization admin user registers their organization with id 12345 to a domain name they specify, domain.org, we would automatically register it in DigitalOcean and requests made to that domain should be mapped to the corresponding organization on our end as follows:

domain.org/ -> oursite.org/organization/12345
domain.org/dashboard -> oursite.org/organization/12345/dashboard
domain.org/about -> oursite.org/organizatin/12345/about

I am not sure if this is a common problem but I am not sure how to set it up. Perhaps the way to go is to set a root handler:

urlpatterns = patterns('',
    (r'^$', root),
    ...
)

And that root view would have the responsibility of looking up the organization id that corresponds to the URL’s domain which can be accessed with request.get_host().

  • is this a valid approach or typically this is done differently?

  • domain.org should behave as if redirected to oursite.org/organization/12345 but without changing the URL. This should work like a reverse proxy, without the client knowing behind the scenes domain.org/whatever actually translates to oursite.org./organization/12345/whatever.

  • how can URLs be rendered using the {% url %} tag so that instead of our site’s domain name, the organization’s domain name is displayed? This is natural: we would want to have links to domain.org/some-page, not oursite.org/organization/12345/some-page.

  • would these domain names need to be added to ALLOWED_HOSTS? Would this mean I need to restart the server every time a user registers a new domain? How do I deal with this?

2

The best approach is to use a middleware to replace request.urlconf, based upon request.headers.get_host() and your DB. You could also attach the organization to the current request there as request.organization, then make your views try to read the organization from there if it’s set, rather than the URL kwarg.

Hope that helps,

Adam

3

Thanks for your reply. I was tinkering with a middleware to set request.domain which would allow me to work with the specific organization in my views. But that’s too “late” in the sense of runtime; I need to work at URL level. I need domain.org/page to stay domain.org/page, not show as oursite.org/organization/12345/page to the client.

Could you elaborate on what I’d need to replace request.urlconf with? Checking out the docs I’m not even sure what I’m looking at here. All I need to do here is to replace it with the custom domain name? (e.g domain.org)? Not sure I understand this bit, I don’t know how to proceed.

4

It’s covered in the linked docs
https://docs.djangoproject.com/en/3.0/topics/http/urls/#how-django-processes-a-request

5

I’m curious here - and just thinking outside the box a little bit - but is there a reason why you need to “internally translate” these URLs?

Is there any reason why the “internal” url can’t be something like oursite.org/domain.org/page? It would seem like an easier translation / mapping process - and perhaps manageable at the webserver / proxy layer rather than within Django.

6

Ok, I see that configuring request.urlconf would change the “urls” file currently being used. But I fail to see how that would help me map domain.org to mysite.com/12345 without redirecting. Please help me understand your suggestion, I’m having a lot of trouble trying to wrap my head around what needs to be done for this feature.

7

I have no sysadmin experience other than what I studied at uni years ago, so I wouldn’t have a clue how to set it up at webserver or proxy layer. I’ve been researching on the topic for a few days and in very abstract terms (in the sense that I wouldn’t know how to start), I’ve considered the possibility of a reverse proxy server with another Django app that would query the database for the corresponding [domain name, organization id] record which then would request the current server oursite.org/<organization_id>. Problems:

  • this is way beyond my skills

  • links generated by Django would point to the internal server rather than the proxy one. I wouldn’t know how to generate links with domain.org based on each organization as a root instead of the usual server oursite.org

  • users will register domains in our system themselves. If we handled this at proxy server level like you say, wouldn’t we have to define rewrite rules manually so that each domain maps to each oursite.org/<organization_id> route?

Maybe a lot of what I say doesn’t make sense. That’s how clueless I am and that’s why I’m looking for help.

8

Thanks for the clarifications - yes, it all makes sense.

Addressing them in order -

  • It might be knowledge you don’t currently have, but I seriously doubt it’s “beyond” you. Learning new things is always fun! :smiley: (Sorry, I can’t be more helpful with this one.)

  • There are generally two ways in which you generate URLs, either using the {% url %} tag or the reverse method. Just off the top of my head, I believe they both return absolute internal urls as strings, in which they could be prefixed with your specific info.

example, where you might normally do something like:

def get_url(arg1, arg2):
    return reverse('page', kwargs={'arg1': arg1, 'arg2': arg2})

could be replaced with:

def get_url(site_id, arg1, arg2):
    site_name = Sites.objects.get(pk=site_id).site_name
    return site_name + reverse('page', kwargs={'arg1': arg1, 'arg2': arg2})

(This idea is why I asked about the possibility of using the site name within the path rather than the organizational id. This type of thing could also be done within your templates to handle the {% url %} case.)

  • Nginx (and Apache IIRC - sorry, I have no direct knowledge of any other web server) both allow regexs for redirects. You could do something like this:
server_name oursite.org;
    if ($host != $server_name) {
        rewrite ^/(.*) $scheme://$server_name/$1 last;
    }

This rewrites the incoming request before passing it into your application
(That’s a bit off-the cuff, I’m not sure it’s 100% correct.)

There might be some details I’m missing here, but this is generally what I was thinking of when I saw your original question.

Ken

9

So basically if I want to reuse existing templates and views I need to rewrite how url works (or use a different tag everywhere) so that the returned URL is the one we’ve always used if we are in oursite.org, but the one with the organization’s domain if we in some other domain.org? I am not sure how feasable this is, it’d mean rewriting so many templates.

The way that nginx rewrite rule works, if I understand correctly, is that it takes the host name URL part (e.g. domain.org) and forms oursite.org/domain.org so that Django’s routes parse it as a kwarg and I handle it appropriately? (in this case find the corresponding organization id)

Please let me know if I am in the right path of understanding the problem.

An additional issue arises. When I say domain.org/page should map to oursite.org/<organization_id>/page, I also mean the same exact output is expected. However I can’t just redirect to the second URL because that’s going to change the URL. I don’t want to redirect. Is it just a matter of rewriting my views to either take an organization kwarg or a domain name kwarg? Either rewriting or extending my CBVs to override the behavior, just slightly. Would that be it?

10

Regarding rewriting url - I’m kinda assuming you’d be doing it anyway because of your requirement to only “advertise” (expose to the outside world) the oursite/domain/page style URLs. Otherwise you’re emitting the internal URLs and would have to do some type of “text scanning” on output to catch the references, and I have my personal doubts that that can be reasonably done.

Before passing it into the application, it would convert a request of the form https://domain.org/some-page to https://oursite.org/domain.org/some-page, and that’s the url being passed into the application.
This way, the originating domain is the top level url component. (Note that this is not a redirect. The “last” parameter means to change the url and rescan for a url match.)

And yes, the intent is that this becomes a kwarg within your view that you can use/ignore at your choosing.

Note - I admit that I’m just kinda winging this off-the-cuff. I have no idea whether this would all work in practice - it’s just the thought that popped into my head. If I were faced with this situation, it’s likely the initial direction I would take. It could turn out that I would end up burning a couple days on a PoC that fails miserably.

More from the sounds of it though, is that it also looks like this might be a candidate for the “sites” framework. That decision may depend upon just how much overlap there is between “private” (domain.org) content and shared / common content provided through oursite.org. I’m getting the impression that this might be less a case of needing to redirect and more a case of just separate content?
I mean, it seems to me that if they’re generating private content, it might just be easiest to serve it from “domain.org/dashboard” rather than “oursite.org/domain.org/dashboard”. In that case, you’re avoiding all the rewrite-related issues.

Ken

11

Thank you. That was really helpful and makes me see a solution to my problem more doable.

Yes, users would add content to their pages, but at the same time we want to show some common (in a sense) content we have in our system.

I have been looking at the sites framework but I fail to see how it’d help me. It sounds more like it’d complicate things for me further, rather than help.

I mean, it seems to me that if they’re generating private content, it might just be easiest to serve it from “domain.org/dashboard” rather than “oursite.org/domain.org/dashboard”. In that case, you’re avoiding all the rewrite-related issues.

Waht makes you say that? I see a couple of problems with serving directly form domain.org/page:

  • domain.org needs to be manually added to ALLOWED_HOSTS and requires a server restart. This just wouldn’t be practical. While you suggestion to map these requests to a common endpoint oursite.org/domain.org page would be accepted no problem.

  • rather, the endpoint would be something like outsite.org/domains/domain.org/page so that I can create a url patterns under /domains/ and handle domain-specific requests in a separate module.

  • having the domain name as a kwarg makes reusing my CBVs more easily than if I had to parse the URL and extract the domain name.

12

Ok, chasing down the “sites” framework idea idea a little deeper:

  • ALLOWED_HOSTS can be set to ‘*’ - allows all host headers.

  • If you want to reject requests from domains not in your list, you could write custom middleware to check the host header yourself.

  • You won’t need to parse the URL yourself, the sites framework passes the site ID in the request object.

  • I don’t know how your views are structured, or the nature of the content they’re providing, but if you’re pulling pages or page data from the database and rendering them through a common template, it seems like it would be easy enough to create queries that filter by the supplied site id, or return the default if there’s no data for that site.

Please understand - I fully recognize that you know your needs and requirements far better than
I ever will. I’m not trying to be argumentative here, and if I’m coming across that way, I apologize. I’m just “spitballing” some ideas in the hopes that something I say resonates with you and helps you find a solution that’s going to work within the environment and context in which you’re working.

But most importantly as part of this disclaimer, I’ve never used the sites framework outside a very limited proof-of-concept demonstration for someone a couple years ago, and that was in the context of multiple subdomains. I have very little direct knowledge of it outside the documentation, so take everything I say here with a large spoonful of salt.

Hope any of this helps.

Ken

13

I understand you. However what you say makes sense and was really helpful. It does apply to my scenario.

I was concerned with setting ALLOWED_HOSTS to any connection because that’s a security issue, but it’s true I can have a middleware taking care of allowing only trusted hosts. Still, I think it’s simpler to stick to your original suggestion and have a rewrite rule that turns domain.org/some-page to oursite.org/endpoint/domain.org/some-page.

My biggest issue now, however, is dealing with URLs generated by the app. {% url %} would use oursite.org as hostname. And whenever the organization id is used as a kwarg, we wouldn’t really want it, but I guess we’ll have to live with it. Anyway, the template tag should output a URL using domain.org instead. You addressed this bit in a couple of responses above but I didn’t quite follow what you said.

14

In thinking about it a little more, I may have had things “backwards” when I was thinking about the {% url %} changes.

Let me see if I’ve got this straight (I think I’m talking through this more for my benefit than yours) ->

Browser issues a request for http://domain.org/page. On your end, it gets translated to http://mysite.org/domain.org/page, which matches to some url domain.org/page.

Your view does stuff, and you want to render a link to http://domain.org/page2. However, internally, {% url 'page2' %} is going to generate /domain.org/page2, which as-is is going to be assumed to be going to http://domain.org/domain.org/page2 since no scheme/host was specified.

So, lets say your template has something like:
<a href="{% url 'page2' %}">link to page 2</a>
you would actually see:
<a href="/domain.org/page2">link to page 2</a>

However, what you probably want is either:
<a href="http://domain.org/page2">link to page 2</a>
or:
<a href="/page2">link to page 2</a>

To accomplish the first:
<a href="http:/{% url 'page2' %}">link to page 2</a>
will prefix the scheme and the extra slash, changing a site-absolute url to a full url.

The other option - dropping the '/domain.org' at the beginning of the URL doesn’t look so easy. (I thought there was a filter that could help with this, but I’m not finding one.)
The easiest way I can think of would be to create your own url template tag that calls the standard url template tag and modifies it before returning it to the template engine. You then would need to load your custom library at the top of every template, but it avoids the need to modify every existing url tag in the template, so you don’t risk missing any individual instances.

(I think, in part, one of the items to consider is the number of views / templates that this affects. My choice might differ depending upon whether I’m looking at 100 views and templates or 1000.)

Ken

P.S. I’m thinking that this is probably the biggest reason why I started thinking of using the sites framework. Your templates and other URL-generators (e.g. reverse) won’t need to be modified because no url translations would be necessary.

15

Yeah, modifying all existing uses of {% url %} is not realistic, that’s not going to work. I should probably use the sites framework then. I’ll ditch the rewrite rule and just stick with domain.org/some-page and have ALLOWED_HOSTS to accept every connection. I can deal with non trusted hosts with a middleware.

However I don’t see what the sites framework has to offer compared to what I can do from just getting the host name from the request. Maybe I don’t need to use Site. I don’t see the benefit, perhaps I’m missing something the framework lets you do.

16

“Modifying existing uses of {% url %} is not realistic” - Agreed. I’d probably be more inclined to go with the modified template in that situation.

Actually, I’m going to withdraw the idea of using the Site framework. Doing some more reading makes me realize it’s not as dynamic-friendly as I seem to remember. (Something that I didn’t pick up on while reading about it earlier is that it expects to run each site under a separate container with a separate settings file - clearly not suitable for what you’re trying to do.)

So yea, I think you’re settling in on what’s likely to be the best solution for what you’re trying to manage.