Cannot stop Invalid HTTP_HOST header in a safe way.

Using Django Deployment
1

Recently deployed a couple of Django apps to my server. One of them is called aicoverlettercreator.com.

As usual when I deploy it to my server, I receive this message to my admin email account. These are all bots trying to ping my server, checking for vulnerabilities etc.

These emails are also extremely annoying. So I would like to block them.

Invalid HTTP_HOST header: 'www.twitch.tv:443'. You may need to add 'www.twitch.tv' to ALLOWED_HOSTS.

DisallowedHost at 443
Invalid HTTP_HOST header: 'www.twitch.tv:443'. You may need to add 'www.twitch.tv' to ALLOWED_HOSTS.

Request Method: CONNECT
Request URL: http://www.twitch.tv:443/443
Django Version: 4.2.2
Python Executable: /usr/local/bin/python
Python Version: 3.10.12
Python Path: ['/code', '/usr/local/bin', '/usr/local/lib/python310.zip', '/usr/local/lib/python3.10', '/usr/local/lib/python3.10/lib-dynload', '/usr/local/lib/python3.10/site-packages']
Server time: Fri, 21 Jul 2023 05:35:10 +0000
...

I’ve tried multiple ways to stop this:

Step 1: Django Settings

Here are the relevant Django settings;

# ... 
ALLOWED_HOSTS = [
    "aicoverlettercreator.com",
    "localhost",
    "127.0.0.1",
    "the.ip.of.my.server",
]
CSRF_TRUSTED_ORIGINS = ["https://aicoverlettercreator.com"]
# ...

This is the normal config, nothing interesting. But kept receiving these emails.

Step 2: Blocking in my webserver

All my web apps are served in a single server using Caddy as a reverse proxy. After seeing multiple posts saying that I should tweak my nginx config, I blocked all hosts who’s header is not aicoverlettercreator.com:

aicoverlettercreator.com {
    @correctHeader {
        header_regexp Host aicoverlettercreator\.com$
    }
    handle @correctHeader {
        reverse_proxy :8888
    }
    @correctHTTPHost {
        header HTTP_HOST aicoverlettercreator.com
    }
    handle @correctHTTPHost {
        reverse_proxy :8888
    }
}

Still receiving the emails.

Step 4: Blocking through Cloudflare

So nothing helped and I decided to block these requests also through Cloudflare:

image
image1084Ă—420 26 KB

But they keep coming!!!

A bit desperate at this point. What else can I do to stop these emails?

2

It appears to me that you should be able to apply a filter in your Django logging for those.

See Logging | Django documentation | Django

Side note: I also use fail2ban to monitor activity and to block “hostile” IP addresses, as well as an appropriate robots.txt file to reduce the crawler load on the site.

3

Yeah, thought that was something of a bad practice. Now that I’ve seen in the docs I’ll just go ahead w/ that.

Crazy that no other solution worked.

4

I’m not familiar with Caddy, but the header as supplied by the browser is “Host”, not “HTTP_HOST”. It’s the python wsgi container (e.g. Werkzeug) that adds the “HTTP_” prefix for Django. I think Caddy would see it as “Host”. (That’s my guess based upon the examples at header (Caddyfile directive) — Caddy Documentation).

5

@KenWhitesell thanks for the info.

As you can see from the post above I actually set BOTH of them and none of them worked :slight_smile:

6

I see that now, thanks!

Is there any sort of “default” action defined? It appears to me that these will handle the cases when the headers match, but it’s not clear to me what’s going to happen if those tests fail.

Is there any way to log these to get more information?

7

Yes I believe so. For example:

aicoverlettercreator.com {
    @correctHeader { # match this host by regex
        header_regexp Host aicoverlettercreator\.com$
    }
    handle @correctHeader { # if a match - redirect to the right port
        reverse_proxy :8888
    }
    @correctHTTPHost { # same but match HTTP_HOST
        header HTTP_HOST aicoverlettercreator.com
    }
    handle @correctHTTPHost { # and redirect 
        reverse_proxy :8888
    }
}

This is pretty much how it should work.

8

Ok, but my question is, what happens if neither of those conditions match? What’s the default operation going to be?

9

Nothing (e.g., the request does not reach django.)

1 Like