Using custom prefixes with urlpatterns

Hi! We have been working to achieve this behavior: Use some prefixes to identify some subsites in our domain which may be in different languages. We would like to get the site identified by the prefix, check the language in our sites dictionary (in settings) and somehow activate the language.

We thought the best approach was some kind of i18n middleware wrapper but we can’t get it to work because resolver seems to skip the populating phase when we have just done it with some language (even if we store them using the prefix as key)

Does anyone here master this part of django so we can have a bit of brainstorming?

Thanks!

Hm, that’s tricky. When you tried the middleware, were you doing it before the view was resolved, or after? That would probably have been my initial approach too, but I’m not quite sure what you mean when you say “resolver seems to skip the populating phase”.

This should be doable in a middleware. Have you looked at how the LocaleMiddleware that ships with Django works?

Essentially, given the new-style middleware format, something along these lines should work:

class MyLocaleMiddleware:
    def __init__(self, get_response=None):
        self.get_response = get_response

    def __call__(self, request):
        language = self.get_lang_for_path(request.path)
        translation.activate(language)
        request.LANGUAGE_CODE = translation.get_language()

        response = self.get_response(request)

        response.setdefault('Content-Language', language)
        return response

    def get_lang_for_path(self, path):
        # Implement your logic here
        return "en"
2 Likes

@andrewgodwin @MarkusH
Actual approach is like this:

MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    # 'django.middleware.locale.LocaleMiddleware',
    'ov2.localizedsites.conf.middleware.LocalizedSitesMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
]  

A ov2.localizedsites.conf.utils.py which offers this functions:

is_site_prefix_patterns_used, get_site_from_request,
activate_site, get_language_from_site, get_site,
get_site_from_path,

The LocalizedSites Middleware

class LocalizedSitesMiddleware(MiddlewareMixin):
    """
    Parse a request, set the site stuff and decide what translation object to install in the
    current thread context. This allows pages to be dynamically translated to
    the language the user desires (if the language is available, of course).
    """

    response_redirect_class = HttpResponseRedirect

    def process_request(self, request):
        urlconf = getattr(request, 'urlconf', settings.ROOT_URLCONF)
        site_prefix_patterns_used = is_site_prefix_patterns_used(urlconf)
        site = get_site_from_request(request, check_path=site_prefix_patterns_used)
        activate_site(site)
        language = get_language_from_site(site)
        translation.activate(language)
        request.LANGUAGE_CODE = translation.get_language()
        request.SITE_PREFIX = site

    def process_response(self, request, response):
        language = translation.get_language()
        site = get_site()
        site_from_path = get_site_from_path(request.path_info)
        urlconf = getattr(request, 'urlconf', settings.ROOT_URLCONF)
        site_prefix_patterns_used = is_site_prefix_patterns_used(urlconf)

        if (
            response.status_code == 404
            and not site_from_path
            and site_prefix_patterns_used
        ):
            site_path = '/%s%s' % (site, request.path_info)
            path_valid = is_valid_path(site_path, urlconf)
            path_needs_slash = not path_valid and (
                settings.APPEND_SLASH
                and not site_path.endswith('/')
                and is_valid_path('%s/' % site_path, urlconf)
            )

            if path_valid or path_needs_slash:
                script_prefix = get_script_prefix()
                # Insert site after the script prefix and before the
                # rest of the URL
                site_url = request.get_full_path(
                    force_append_slash=path_needs_slash
                ).replace(script_prefix, '%s%s/' % (script_prefix, site), 1)
                return self.response_redirect_class(site_url)

        if not (site_prefix_patterns_used and site_from_path):
            patch_vary_headers(response, ('Accept-Language',))
        response.setdefault('Content-Language', language)
        response.set_cookie(
            settings.SITE_COOKIE_NAME,
            site,
            max_age=settings.SITE_COOKIE_AGE,
            path=settings.SITE_COOKIE_PATH,
            domain=settings.SITE_COOKIE_DOMAIN,
            samesite='Strict',
        )
        return response

There is also our custom urlpatterns generator:

def localizedsites_patterns(*urls):
    """
    Add the localizedsite prefix to every URL pattern within this function.
    This may only be used in the root URLconf, not in an included URLconf.
    """
    if not settings.USE_LOCALIZED_SITES:
        return list(urls)
    return [SiteURLResolver(LocalizedSitePrefixPattern(), list(urls))]

which is used in all the url entries that we want to work this way.

And finally, in our resolvers.py:

class LocalizedSitePrefixPattern:
    def __init__(self):
        self.converters = {}

    @property
    def regex(self):
        # This is only used by reverse() and cached in _reverse_dict.
        return re.compile(self.site_prefix)

    @property
    def site_prefix(self):
        site_prefix = get_site()
        return '%s/' % site_prefix

    def match(self, path):
        site_prefix = self.site_prefix
        if path.startswith(site_prefix):
            return path[len(site_prefix) :], (), {}
        return None

    def check(self):
        return []

    def describe(self):
        return "'{}'".format(self)

    def __str__(self):
        return self.site_prefix

class SiteURLResolver(URLResolver):
    def _populate(self):
        # Short-circuit if called recursively in this thread to prevent
        # infinite recursion. Concurrent threads may call this at the same
        # time and will need to continue, so set 'populating' on a
        # thread-local variable.
        if getattr(self._local, 'populating', False):
            return
        try:
            self._local.populating = True
            lookups = MultiValueDict()
            namespaces = {}
            apps = {}
            site_prefix = get_site()
            for url_pattern in reversed(self.url_patterns):
                p_pattern = url_pattern.pattern.regex.pattern
                if p_pattern.startswith('^'):
                    p_pattern = p_pattern[1:]
                if isinstance(url_pattern, URLPattern):
                    self._callback_strs.add(url_pattern.lookup_str)
                    bits = normalize(url_pattern.pattern.regex.pattern)
                    lookups.appendlist(
                        url_pattern.callback,
                        (bits, p_pattern, url_pattern.default_args, url_pattern.pattern.converters)
                    )
                    if url_pattern.name is not None:
                        lookups.appendlist(
                            url_pattern.name,
                            (bits, p_pattern, url_pattern.default_args, url_pattern.pattern.converters)
                        )
                else:  # url_pattern is a URLResolver.
                    url_pattern._populate()
                    if url_pattern.app_name:
                        apps.setdefault(url_pattern.app_name, []).append(url_pattern.namespace)
                        namespaces[url_pattern.namespace] = (p_pattern, url_pattern)
                    else:
                        for name in url_pattern.reverse_dict:
                            for matches, pat, defaults, converters in url_pattern.reverse_dict.getlist(name):
                                new_matches = normalize(p_pattern + pat)
                                lookups.appendlist(
                                    name,
                                    (
                                        new_matches,
                                        p_pattern + pat,
                                        {**defaults, **url_pattern.default_kwargs},
                                        {**self.pattern.converters, **url_pattern.pattern.converters, **converters}
                                    )
                                )
                        for namespace, (prefix, sub_pattern) in url_pattern.namespace_dict.items():
                            current_converters = url_pattern.pattern.converters
                            sub_pattern.pattern.converters.update(current_converters)
                            namespaces[namespace] = (p_pattern + prefix, sub_pattern)
                        for app_name, namespace_list in url_pattern.app_dict.items():
                            apps.setdefault(app_name, []).extend(namespace_list)
                    self._callback_strs.update(url_pattern._callback_strs)
            self._namespace_dict[site_prefix] = namespaces
            self._app_dict[site_prefix] = apps
            self._reverse_dict[site_prefix] = lookups
            self._populated = True
        finally:
            self._local.populating = False
        print('#'*20,'_populate', site_prefix)

    @property
    def reverse_dict(self):
        site_prefix = get_site()
        print('reverse_dict', site_prefix)
        if site_prefix not in self._reverse_dict:
            self._populate()
        print(self._reverse_dict)
        return self._reverse_dict[site_prefix]

    @property
    def namespace_dict(self):
        site_prefix = get_site()
        print('namespace_dict', site_prefix)

        if site_prefix not in self._namespace_dict:
            self._populate()
        print(self._namespace_dict)
        return self._namespace_dict[site_prefix]

    @property
    def app_dict(self):
        site_prefix = get_site()
        print('app_dict', site_prefix)

        if site_prefix not in self._app_dict:
            self._populate()
        print(self._app_dict)
        return self._app_dict[site_prefix]

Let’s assume that we have two sites with the same language:
Site | Prefix | Lang
1 | mad | es-es
2 | bcn | es-es
3 | nyc | en-us

Not sure how to use Python syntax highlight here :frowning:

The Site prefix detection and that seems to works:
We’ve got
reverse_dict, namespace_dict and app_dict to store urls using ‘prefix’ as the dictionary key

If I do the request localhost:8000/mad/ (home view) it works and it populates namespace_dict this way:

{'mad': {'article': ('', <URLResolver <module 'ov2.ocms.urls' from '/Users/mike/development/ov2/ov2/ocms/urls.py'> (ocms:article) ''>), 'admin': ('admin/', <URLResolver <URLPattern list> (admin:admin) '^admin/'>)}

Then if ask for localhost:8000/nyc/ it seems to work fine:

{'mad': {'article': ('', <URLResolver <module 'ov2.ocms.urls' from '/Users/mike/development/ov2/ov2/ocms/urls.py'> (ocms:article) ''>), 'admin': ('admin/', <URLResolver <URLPattern list> (admin:admin) '^admin/'>)}, 'nyc': {'article': ('', <URLResolver <module 'ov2.ocms.urls' from '/Users/mike/development/ov2/ov2/ocms/urls.py'> (ocms:article) ''>), 'admin': ('admin/', <URLResolver <URLPattern list> (admin:admin) '^admin/'>)}

But when I do ask for localhost:8000/bcn/ no new entries are added in those dicts.
Not sure if that’s why reverse (called by url templatetag) sticks to first spanish set of urls generated, so that relative links (while browsing /bcn/ site) are written with the /mad/.

If I restart the server and visit bcn and mad sites in different order I get the bcn prefix when when reversing.

Any ideas?? Thanks!

Why do you have such a big URLResolver override? I would have approached this with a middleware which merely swaps in different urlconfs by setting request.urlconf, which is the supported way of doing this; I worry what you’re doing is too complex and you’re somehow breaking internal caching order.

We were trying to change what URLResolver’s _populate() does:

self._namespace_dict[language_code] = namespaces
self._app_dict[language_code] = apps
self._reverse_dict[language_code] = lookups

into:

self._namespace_dict[site_prefix] = namespaces
self._app_dict[site_prefix] = apps
self._reverse_dict[site_prefix] = lookups

I’m not sure how urlconf swap could help us, we did really want something like:

urlpatterns = localizedsites_patterns(
    re_path(r'', include('ov2.ocms.urls', namespace='article')),

in our project.
Do you mean we should spread those patterns into multiple urlconfs?

Using multiple urlconfs is traditionally what I’ve done. Have you tried pre-generating the various urlconfs with the locale prefixes in them? It takes up a bit more memory but the result is you’ll have one urlconf per locale you can easily swap in rather than trying to override the whole resolution mechanism.