Why is the use of `url_has_allowed_host_and_scheme` discouraged

Using Django
1

Hello Djangonauts.

First time poster here :wave:.

I have been looking for a way to check that a redirect URL is “safe” (e.g. local to the current site). I have hand rolled a function to only leave the path without the domain. Then I thought this must be a common problem, there must be something in Django.

Turns out there was is_safe_url, which was renamed to url_has_allowed_host_and_scheme. However, both are undocumented and only mentioned in the release notes. And when they are mentioned it does not sound like you are supposed to be using them.

Why is that?

Then there is this quote from the 4.2 release notes:

This is to protect projects that may be incorrectly using the internal url_has_allowed_host_and_scheme() function, instead of using one of the documented functions for handling URL redirects.

— Django 4.2 release notes | Django documentation | Django

The most interesting part is “instead of using one of the documented functions for handling URL redirects”. I tried but failed to find the documented functions for handling URL redirects. Does anyone have any pointers what those might be?

1 Like
2

url_has_allowed_host_and_scheme is an internal function because it only does part of what’s needed to safely use a provided URL in a redirect. There’s escaping issues and other considerations, all of which need to be applied, and for which it’s easy to miss.

Before the change you link the function was called is_safe_url — which gave folks entirely the wrong impression: they’d use that — despite the warnings that say don’t — and think they’d done enough.

The release note is only there as a courtesy to those folks.

Again this is an internal function. Don’t use it.

If you want to do redirects you should always use the documented HttpResponseRedirect which correctly handles the various issues here for you.

1 Like
3

Thanks for your response @carltongibson.

I am not sure I understand how HttpResponseRedirect will make sure that the redirect is safe. Wouldn’t it happily redirect to an unsafe site if one used something like HttpReponseRedirect(url=request.GET["next"]) (ignoring that the key could be missing of course)?

4

I agree, the rename to url_has_allowed_host_and_scheme makes it much clearer of what the function does. If a URL that passes those checks can be considered safe would depend on the given context.

I understand that the function is meant for internal use and not part of the documented public API of Django.

What I don’t quite understand is: why?

It seems like a perfectly valid utility function that does what it says.

It is used exactly how you would expect in the django.auth.views module to check the “next” parameter.

What I don’t understand is what would make using the function elsewhere in the exact same manner (to check a URL coming from a query parameter) so worrisome?

EDIT: I guess my question is: Why is it considered “private and internal”?

5

The code you quote there goes on to use the redirect URL with an HttpResponseRedirect. The latter then performs required escaping before using the URL. It’s only in such combination that a URL is safe.

Django doesn’t provide equivalent utilities as part of the public API because doing so in a generic way but such that folks won’t step into security problems has not been considered feasible.

6

Thanks for expanding on this @carltongibson.

The URL escaping you mention, is that the typical % escaping of potential query parameters in the URL which was checked to be of allowed host and scheme? We would not want to escape the whole URL right, because then it wouldn’t work as a URL?

Similar to what’s mentioned on this cheat sheet?

7

Ah, OK. You want the full details. That will require me to go over the history. I’m happy to do that on the back burner, for my own refresh, but it won’t be instant I’m afraid.

8

I would definitely be happy about anything that helps me understand this fully.

Sorry for the late response I did not get a notification for some reason.

9

OK, no problem. I’ll see if I can write something up. (No deadline! :sweat_smile:)

tl;dr will be ≈"folks still need to correctly escape the URL, and that’s non-obvious"

10

Thanks @carltongibson. That would be much appreciated.

11

Hi everyone!

I was wondering about this issue as well.

Again this is an internal function. Don’t use it.

If you want to do redirects you should always use the documented HttpResponseRedirect which correctly handles the various issues here for you.

Ok, I have some questions about this:
If I have this view:


def login(request):
    if request.user.is_authenticated:
        return redirect("home")

    if request.method == "POST":
        form = EmailLoginForm(request, data=request.POST)

        if form.is_valid():
            user = form.get_user()
            login(request, user)
 
            next_url = request.GET.get("next")

            if next_url:
                return redirect(next_url)

            return redirect("home")

    else:
        form = EmailLoginForm()

    return render(request, "login.html", {"form": form})

How would you manage next_url?

  1. Maybe it is not necessary? Or is there another way to redirect to the url the user came form?
  2. I thought that I could use url_has_allowed_host_and_scheme, but @carltongibson said it’s for Django’s internals and not a public API.
  3. Does redirect manage this? I didn’t find it in the source code.

@carltongibson Have you written something up?:winking_face_with_tongue:

Thanks!

12

No, I haven’t. Time, alas. :person_juggling:

The bottom line is this API is tricky to use correctly, safely. If Django documents it as public API, folks will use it incorrectly, expose themselves to security issues, and blame Django — no matter what warnings are in place. That doesn’t correspond to Django’s safe by default goals. (And I don’t really see a way to square that circle.)

If you think you understand all the issues, and can trust yourself and your team to use these methods safely and without mistake, go ahead of course. But caveat emptor — the risk there is on you.

1 Like
13

Thanks Carlton!

If you don’t mind asking, how do you manage redirects safely with things like next_url = request.GET.get("next").

:winking_face_with_tongue:

14

The redirect shortcut function returns an HttpResponseRedirect object as suggested.

1 Like
15

Hi Ken,

Yes, I understand.

But does HttpResponseRedirect handle safely the redirecting to a malicious URL?

Thanks.

16

In part, this depends upon what you consider a “malicious URL” to be.

As far as this thread is concerned regarding url_has_allowed_host_and_scheme, it was answered above:

If you’re talking about a different situation, then I would suggest you open a new topic for that discussion.

(And, of course, if you have specific examples, you can always test these using the Django shell to see what those functions return.)

1 Like
17

Thanks @KenWhitesell and @carltongibson for the follow-ups.

To summarize, would it then be fair to say the following snippet represents a safe use of the url_has_allowed_host_and_scheme and redirect functions?

def my_view(request):
    next_url = request.GET.get('next')
    
    if next_url and url_has_allowed_host_and_scheme(next_url):
        return redirect(next_url)
    
    return redirect("/")
1 Like
18

This is not necessary. As quoted above:

[Emphasis added]

If you can demonstrate an example where this does not work as intended, then it’s likely to be a bug that would need to be fixed.

1 Like
19

I don’t think that’s true. url_has_allowed_host_and_scheme definitely has a place here. Accepting a query param of a URL and redirecting to it without validation is a security vulnerability: Open redirect vulnerability | Tutorials & examples | Snyk Learn.

HttpResponseRedirect (and the redirect shortcut which returns it) does harden the redirect process (more so than building the Location header yourself), specifically on the response side. When it comes to validating where to redirect the user, that’s where url_has_allowed_host_and_scheme (and others) comes in. Any input coming from the user must be validated. Checking input is a URL is (relatively) easy, or at least Django already has a public API to do it. But checking that you’re about to redirect the user somewhere they’re allowed to is still a necessary part of the process. In many cases, you may only want to accept relative redirect URLs, which is fine and not too hard to implement.

Personally, both with and without my Security team hat on, I could definitely see the value in url_has_allowed_host_and_scheme being part of the public API, or even being run as part of the redirect responses (probably not though, it’d probably be very breaking). Or, if it’s kept internal, a unified API which handles all the necessary validation (there’s not that much, but missing one makes the rest useless) and does a redirect.

Edit: I’ve now raised a suggestion on the new-features repo: Harden `django.shortcuts.redirect` to validate redirect URL · Issue #97 · django/new-features · GitHub

I completely agree with @carltongibson that it’s not the full story when it comes to safely handling URLs, but seeing as it’s a critical piece, and one Django already does internally as standard in say the admin, it makes sense to me to expose that functionality publicly. Once it is, we could (and probably should) also do more to document how to redirect securely (validate host, scheme changes, iri_to_uri etc), rather than leaving it up to the user to work out (as @tbrlpld has had to).

1 Like
20

I wouldn’t handle a next parameter to an unknown URL. I’m going to know the allowed targets ahead of time, and the “is this allowed” logic is much less generic than Django’s internal handling needs to be. (TBH I don’t see why folks want something fully generic here, which is always going to be hard to audit: just write the function you need.)

I then use redirect response to ensure the proper escaping.

It’s pretty simple I think. :person_shrugging:

1 Like