Bad Request and Invalid HTTP_HOST header in deployment NGINX + Gunicorn + Django?

Could you, please, help me with some suggestion where or how to find a resolution of my DisallowedHost exception?

I have deployed my Django project to the DigitalOcean Ubuntu server with Nginx + Gunicorn according to instructions from DjangoProject and DigitalOcean as well as answers from related questions from DjangForum, StackOverflow and others. I believe I tried all the advice I could find but still have 400 BadRequest responses, and Invalid HTTP_HOST header Error.

likarnet is my project name, and likarnet.com is my domain.
I have connected domain likarnet.com to my public IP and SSL certificate to my domain.
Versions python 3.12, django 5.0.6

Please, excuse me if I ask some obvious things. This is my first project.

File /etc/nginx/sites-available/likarnet

server {
        listen 80 default_server;
        listen [::]:80 default_server;

        server_name likarnet www.likarnet;

        access_log  /var/log/nginx/access.log;
        error_log  /var/log/nginx/error.log;

        location = /favicon.ico { access_log off; log_not_found off; }
        location /static/ {
            root /home/likarnet/website/likarnet;
        }

        location /media/ {
            root /home/likarnet/website/likarnet;
        }

        location / {
            proxy_set_header Host $host;
            proxy_set_header X-Forwarded-Host $server_name;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto https;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_redirect off;
            proxy_pass http://unix:/run/gunicorn.sock;
        }
}

I have just tried to use default include proxy_params;

File /etc/systemd/system/gunicorn.service

[Unit]
Description=gunicorn daemon
Requires=gunicorn.socket
After=network.target

[Service]
User=likarnet
Group=sudo
WorkingDirectory=/home/likarnet/website/likarnet
ExecStart=/home/likarnet/venv/bin/gunicorn \
          --access-logfile - \
          --workers 3 \
          --bind unix:/run/gunicorn.sock \
          --chdir=/home/likarnet/website/likarnet \
          likarnet.wsgi:application

[Install]
WantedBy=multi-user.target

So, when I run project with python3 manage.py runserver 127.0.0.1:8000 or 0.0.0.0:8000 and try from another terminal curl -v localhost:8000 I get

* Host localhost:8000 was resolved.
* IPv6: ::1
* IPv4: 127.0.0.1
*   Trying [::1]:8000...
* connect to ::1 port 8000 from ::1 port 59232 failed: Connection refused
*   Trying 127.0.0.1:8000...
* Connected to localhost (127.0.0.1) port 8000
> GET / HTTP/1.1
> Host: localhost:8000
> User-Agent: curl/8.5.0
> Accept: */*
> 
< HTTP/1.1 301 Moved Permanently
< Date: Fri, 21 Jun 2024 21:43:57 GMT
< Server: WSGIServer/0.2 CPython/3.12.3
< Content-Type: text/html; charset=utf-8
< Location: https://localhost:8000/
< X-Content-Type-Options: nosniff
< Referrer-Policy: same-origin
< Cross-Origin-Opener-Policy: same-origin
< Connection: close
< 
* Closing connection

And in terminal with project I got additional rows Host, ALLOWED_HOSTS, Domain and Port as I added printing them into function get_host() in /venv/lib/python3.12/site-packages/django/http/request.py

Host:  localhost:8000
ALLOWED_HOSTS:  ['likarnet.com', '.likarnet.com', '64.225.77.248', '127.0.0.1', 'localhost']
Domain:  localhost
Port:  8000
[21/Jun/2024 21:51:17] "GET / HTTP/1.1" 301 0

When I try curl --unix-socket /run/gunicorn.sock localhost I get

<!doctype html>
<html lang="en">
<head>
  <title>Bad Request (400)</title>
</head>
<body>
  <h1>Bad Request (400)</h1><p></p>
</body>
</html>

And I get from browser when I input likarnet.com

This site can’t be reached
likarnet.com refused to connect.
ERR_CONNECTION_REFUSED

And the ERROR Traceback from my log file. It is the same when I use browser or curl command, but with various host names - localhost, my public IP, likarnet.com or www.likarnet.com

ERROR 2024-06-21 21:51:06,961 /home/likarnet/venv/lib/python3.12/site-packages/django/core/handlers/exception.py  11561  131208492179584 Invalid HTTP_HOST header: 'localhost'. You may need to add 'localhost' to ALLOWED_HOSTS.
Traceback (most recent call last):
  File "/home/likarnet/venv/lib/python3.12/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/home/likarnet/venv/lib/python3.12/site-packages/django/utils/deprecation.py", line 133, in __call__
    response = self.process_request(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/likarnet/venv/lib/python3.12/site-packages/django/middleware/security.py", line 28, in process_request
    host = self.redirect_host or request.get_host()
                                 ^^^^^^^^^^^^^^^^^^
  File "/home/likarnet/venv/lib/python3.12/site-packages/django/http/request.py", line 151, in get_host
django.core.exceptions.DisallowedHost: Invalid HTTP_HOST header: 'localhost'. You may need to add 'localhost' to ALLOWED_HOSTS.

I am not sure which data is also important. And I am going to add everything needed.
Thanks for your help and advice.

The server name needs to be the full DNS name: likarnet.com and www.likarnet.com See the docs at Module ngx_http_core_module

Also, since you’re proxying this behind nginx, you don’t want to refer to the connection at port 8000. You want to access the site through the nginx port, 80.

Additionally, running another copy of your project on port 8000, when your production version is running behind nginx, doesn’t provide any useful diagnostic information.

It’s going to help in the future here if you stick with one issue - which I assume to be the version running with gunicorn behind nginx. Don’t confuse the issue by bringing in attempts to diagnose this that aren’t related to that.

1 Like

I am sorry for publishing useless information.
After fixing the row following:

When using curl -v localhost:80 I get the same BadGateway

* Host localhost:80 was resolved.
* IPv6: ::1
* IPv4: 127.0.0.1
*   Trying [::1]:80...
* Connected to localhost (::1) port 80
> GET / HTTP/1.1
> Host: localhost
> User-Agent: curl/8.5.0
> Accept: */*
> 
< HTTP/1.1 400 Bad Request
< Server: nginx/1.24.0 (Ubuntu)
< Date: Sat, 22 Jun 2024 06:41:09 GMT
< Content-Type: text/html; charset=utf-8
< Transfer-Encoding: chunked
< Connection: keep-alive
< X-Content-Type-Options: nosniff
< Referrer-Policy: same-origin
< Cross-Origin-Opener-Policy: same-origin
< 

<!doctype html>
<html lang="en">
<head>
  <title>Bad Request (400)</title>
</head>
<body>
  <h1>Bad Request (400)</h1><p></p>
</body>
</html>
* Connection #0 to host localhost left intact

But Traceback is a little different:

ERROR 2024-06-22 06:26:47,521 /home/likarnet/venv/lib/python3.12/site-packages/django/core/handlers/exception.py  124 Func: response_for_exception Task: None Process:  1274 MainProcess Thread:  133201422356608 MainThread Message: Invalid HTTP_HOST header: 'localhost'. You may need to add 'localhost' to ALLOWED_HOSTS. StackInfo: None
Traceback (most recent call last):
  File "/home/likarnet/venv/lib/python3.12/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/home/likarnet/venv/lib/python3.12/site-packages/django/utils/deprecation.py", line 133, in __call__
    response = self.process_request(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/likarnet/venv/lib/python3.12/site-packages/django/middleware/common.py", line 48, in process_request
    host = request.get_host()
           ^^^^^^^^^^^^^^^^^^
  File "/home/likarnet/venv/lib/python3.12/site-packages/django/http/request.py", line 162, in get_host
    raise DisallowedHost(msg)
django.core.exceptions.DisallowedHost: Invalid HTTP_HOST header: 'localhost'. You may need to add 'localhost' to ALLOWED_HOSTS.

Could I have help what can I do next to find a solution?

What if you try curl likarnet.com?

Also, what are the contents of your wsgi.py file?

1 Like

If I use curl likarnet.com I get 400 Bad Request:

<!doctype html>
<html lang="en">
<head>
  <title>Bad Request (400)</title>
</head>
<body>
  <h1>Bad Request (400)</h1><p></p>
</body>
</html>

If try curl -v likarnet.com I get:

* Host likarnet.com:80 was resolved.
* IPv6: (none)
* IPv4: 64.225.77.248
*   Trying 64.225.77.248:80...
* Connected to likarnet.com (64.225.77.248) port 80
> GET / HTTP/1.1
> Host: likarnet.com
> User-Agent: curl/8.5.0
> Accept: */*
> 
< HTTP/1.1 400 Bad Request
< Server: nginx/1.24.0 (Ubuntu)
< Date: Sat, 22 Jun 2024 19:59:56 GMT
< Content-Type: text/html; charset=utf-8
< Transfer-Encoding: chunked
< Connection: keep-alive
< X-Content-Type-Options: nosniff
< Referrer-Policy: same-origin
< Cross-Origin-Opener-Policy: same-origin
< 

<!doctype html>
<html lang="en">
<head>
  <title>Bad Request (400)</title>
</head>
<body>
  <h1>Bad Request (400)</h1><p></p>
</body>
</html>
* Connection #0 to host likarnet.com left intact

File wsgi.py:

import os
from django.core.wsgi import get_wsgi_application

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'likarnet.settings')

application = get_wsgi_application()

What are the contents of your nginx access and error logs for this? What about your gunicorn log?

1 Like

Nginx access log:

64.225.77.248 - - [23/Jun/2024:01:36:22 +0000] "GET / HTTP/1.1" 400 154 "-" "curl/8.5.0"
64.225.77.248 - - [23/Jun/2024:01:36:29 +0000] "GET / HTTP/1.1" 400 154 "-" "curl/8.5.0"
64.225.77.248 - - [23/Jun/2024:01:37:55 +0000] "GET / HTTP/1.1" 400 154 "-" "curl/8.5.0"
64.225.77.248 - - [23/Jun/2024:01:37:59 +0000] "GET / HTTP/1.1" 400 154 "-" "curl/8.5.0"

Nginx error log is empty. I cannot be sure but I think it is set correctly error_log /var/log/nginx/error.log;

Gunicorn access log:

- - [23/Jun/2024:02:10:36 +0000] "GET / HTTP/1.0" 400 143 "-" "curl/8.5.0"
- - [23/Jun/2024:02:10:38 +0000] "GET / HTTP/1.0" 400 143 "-" "curl/8.5.0"

Gunicorn error log has no entries for these actions.

I’m feeling like I’m missing something here, but I’m not sure what.

What are the contents of your /etc/nginx/sites-enabled directory? Is the link to likarnet the only entry? (Do you have any other files linked in that directory that may be causing a configuration conflict?)

What is the output of a systemctl status gunicorn command?

Do you see the gunicorn.sock “file” in the /run directory? What is the output of ls -l /run/gunicorn.sock ?

Your --access-logfile - parameter says that it will be writing the access log to stdout - does that mean it’s going to syslog? What syslog entries do you have for gunicorn? (I would think you’d be better off replacing this with a separate file, along with using the --capture-output and --log-file parameters.) I’d suggest making these changes, restarting the service, restarting nginx and trying this again.

1 Like

I am facing exact same errors and issue when I deployed the environment on Elastic Beanstalk. The response provided by Elastic Beanstalk environment is Target.ResponseCodeMismatch implying that health checks did not return an expected HTTP code.

I don’t think its a solution but more like a patch that fixes the issue temporarily. You can add ’ likarnet.com’ and ‘www.likarnet.com’ in allowed hosts and that shall fix the issue temporarily.

The reason I say temporarily is because ping to the website sends request timed out.

1 Like

Hi, sir!
Thanks for your help!

sites-enabled contains only one link to my website:

# ls -lah /etc/nginx/sites-enabled/
total 8.0K
drwxr-xr-x 2 root root 4.0K Jun 16 12:32 .
drwxr-xr-x 8 root root 4.0K Jun 16 11:10 ..
lrwxrwxrwx 1 root root   35 Jun 16 12:28 likarnet -> /etc/nginx/sites-available/likarnet

I have deleted the default link. Actually I am not sure it is a good idea. But default is still in site-available, so I can recreate it.

gunicorn and gunicorn.socket status:

# systemctl status gunicorn
● gunicorn.service - gunicorn daemon
     Loaded: loaded (/etc/systemd/system/gunicorn.service; enabled; preset: enabled)
     Active: active (running) since Sun 2024-06-23 02:26:05 UTC; 2 days ago
TriggeredBy: ● gunicorn.socket
   Main PID: 12483 (gunicorn)
      Tasks: 4 (limit: 508)
     Memory: 183.0M (peak: 192.2M)
        CPU: 53.380s
     CGroup: /system.slice/gunicorn.service
             ├─12483 /home/likarnet/venv/bin/python /home/likarnet/venv/bin/gunicorn --access-logfile gunicorn_a>
             ├─12484 /home/likarnet/venv/bin/python /home/likarnet/venv/bin/gunicorn --access-logfile gunicorn_a>
             ├─12485 /home/likarnet/venv/bin/python /home/likarnet/venv/bin/gunicorn --access-logfile gunicorn_a>
             └─12486 /home/likarnet/venv/bin/python /home/likarnet/venv/bin/gunicorn --access-logfile gunicorn_a>

Jun 25 08:30:24 ubuntu-s-1vcpu-512mb-10gb-ams3-01 gunicorn[12485]: Port:
Jun 25 08:30:24 ubuntu-s-1vcpu-512mb-10gb-ams3-01 gunicorn[12485]: Host:  64.225.77.248
Jun 25 08:30:24 ubuntu-s-1vcpu-512mb-10gb-ams3-01 gunicorn[12485]: ALLOWED_HOSTS:  ['None']
Jun 25 08:30:24 ubuntu-s-1vcpu-512mb-10gb-ams3-01 gunicorn[12485]: Domain:  64.225.77.248
Jun 25 08:30:24 ubuntu-s-1vcpu-512mb-10gb-ams3-01 gunicorn[12485]: Port:
Jun 25 08:30:24 ubuntu-s-1vcpu-512mb-10gb-ams3-01 gunicorn[12485]: Host:  64.225.77.248
Jun 25 08:30:24 ubuntu-s-1vcpu-512mb-10gb-ams3-01 gunicorn[12485]: ALLOWED_HOSTS:  ['None']
Jun 25 08:30:24 ubuntu-s-1vcpu-512mb-10gb-ams3-01 gunicorn[12485]: Domain:  64.225.77.248
Jun 25 08:30:24 ubuntu-s-1vcpu-512mb-10gb-ams3-01 gunicorn[12485]: Port:
Jun 25 08:30:24 ubuntu-s-1vcpu-512mb-10gb-ams3-01 gunicorn[12485]: Host:  likarnet.com
lines 1-24/24 (END)
# systemctl status gunicorn.socket
● gunicorn.socket - gunicorn socket
     Loaded: loaded (/etc/systemd/system/gunicorn.socket; enabled; preset: enabled)
     Active: active (running) since Sun 2024-06-23 02:26:01 UTC; 2 days ago
   Triggers: ● gunicorn.service
     Listen: /run/gunicorn.sock (Stream)
     CGroup: /system.slice/gunicorn.socket

Jun 23 02:26:01 ubuntu-s-1vcpu-512mb-10gb-ams3-01 systemd[1]: Listening on gunicorn.socket - gunicorn socket.

gunicorn.sock file:

# file /run/gunicorn.sock
/run/gunicorn.sock: socket

or

# ls -l /run/gunicorn.sock
srw-rw-rw- 1 root root 0 Jun 23 02:26 /run/gunicorn.sock

And about --access-logfile. I did not know about gunicorn logs earlier((
When you asked about that I have added rows to gunicorn.service:

--access-logfile gunicorn_access.log \
--error-logfile gunicorn_error.log \

But I do not know how to use --capture-output. When I add --capture-output True \ gunicorn shows error. When I curl likarnet.com - gunicorn_access.log contains:

 - - [25/Jun/2024:09:31:39 +0000] "GET / HTTP/1.0" 400 143 "-" "curl/8.5.0"
 - - [25/Jun/2024:09:31:41 +0000] "GET / HTTP/1.0" 400 143 "-" "curl/8.5.0"
 - - [25/Jun/2024:09:31:42 +0000] "GET / HTTP/1.0" 400 143 "-" "curl/8.5.0"
 - - [25/Jun/2024:09:31:42 +0000] "GET / HTTP/1.0" 400 143 "-" "curl/8.5.0"

Thanks!

My settings.py now load ALLOWED_HOSTS from environment variable.
I have the same error even if I set this parameter directly in settings.py.
It looks like the error appear before ALLOWED_HOSTS are loaded.
But I do not understand all hints how whole system works.

That’s generally the right thing to do. You might want to review what is in that file to determine if there are any settings in it that should be copied to your file.

That’s clearly wrong.

What are the contents of the ALLOWED_HOSTS setting in the /home/likarnet/website/likarnet/likarnet/settings.py file?

1 Like

Thank you very much, sir!
I think, now I understand much more what is wrong, but I do not know how to fix it.
In my settings.py I have the row ALLOWED_HOSTS = list(str(os.environ.get('DJANGO_ALLOWED_HOSTS')).split(';'))

First, I used export DJANGO_ALLOWED_HOSTS=likarnet.com;.likarnet.com;64.225.77.248;127.0.0.1;localhost in /etc/environment file.

Then I tried to use .env file and EnvironmentFile=/home/likarnet/website/likarnet/.env setting.

But both don’t work((

I might not be very much helpful here. But .env file wouldn’t work, atleast in my case it didn’t. You do need to define variable in your OS configuration as you have mentioned.

In case of DJANGO_ALLOWED_HOSTS, I think you will need to cross verify if the output you are getting is correct and stored properly. Try testing it with .env file first and verify values and datatypes using same lines of code you are applying to .env file.

1 Like

If I run gunicorn --bind 0.0.0.0:8000 likarnet.wsgi directly I get printed loaded settings.ALLOWED_HOSTS from command export DJANGO_ALLOWED_HOSTS=likarnet.com;.likarnet.com;64.225.77.248;127.0.0.1;localhost in the file /etc/environment as ALLOWED_HOSTS= ['likarnet.com', 'www.likarnet.com', '64.225.77.248', '127.0.0.1', 'localhost'].

But if I start gunicorn service I get ALLOWED_HOSTS: ['None'].
And I have no ideas why (((

For testing purposes, to verify that the rest of your deployment is correct, I’d replace this with a literal.

What I’d probably do is create a copy of the settings file, and replace all environment variable references with their desired values, and set your wsgi file to refer to that modified version.

(This is a situation of trying to reduce the number of variable in the category of “what can go wrong”.)

Also, once that is working, I suggest you select one method for setting your environment variables, and we can work toward getting that method working. It’s too difficult to try and resolve an issue when you’re moving among different methods.

Also, in the future, for all references to code, errors, configurations, etc, please copy / paste the actual file or output into your message. Do not try to summarize or rephrase. Providing descriptions of what you have done are less helpful than showing what was actually done.

1 Like

Following instructions I set 5 values in settings2.py

SECRET_KEY = 'secret_string'
DATABASE_NAME = 
DATABASE_USER = 
DATABASE_PASSWORD = 
ALLOWED_HOSTS =

Changed the row in wsgi.py:
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'likarnet.settings2')

Then I restarted gunicorn. I am not sure it was necessary but I did.

My browser still refuses: likarnet.com refused to connect.
But when I curl likarnet.com in the terminal - I get expected page html.

And gunicorn access log shows:
- - [25/Jun/2024:13:22:08 +0000] "GET / HTTP/1.0" 200 6308 "-" "curl/8.5.0"

Actually now I do not understand. I have changed that row in wsgi.py file, and it looks like it works there.
At the same moment there is the environment variable export DJANGO_SETTINGS_MODULE=likarnet.settings set in the file /etc/environment. And it does not work, does it? But why?

Are you doing both of these on the server or on a different system, such as your pc?

1 Like

Browser is on my local machine.
curl likarnet.com - from server terminal.

Sorry, I did not expect this is important

It is, because there are potential issues with routing and dns resolution.

On the server, what operating system (and version) are you using?
What happens if you specify the ip address in the address bar? Are you sure you’re issuing an http:// request and not https://?

1 Like