Tweaked settins and configurations too much?

To make my django web project run, I may or may not have made a blunder in production environment. I am already way past the deadline. Earlier the site was working but static content was not displaying. So after making a few changes, all I am getting is different errors everytime. I installed cerbot for ssl. That may have also been causing a lot of issues.

settings.py

mport os
from pathlib import Path

BASE_DIR = Path(__file__).resolve().parent.parent

SECRET_KEY = 'django-insecure-%z88$&huycw6@u%i7-0e#h^_3+ffna$^uk+fvsm=9&=-6rsnz$'

DEBUG = True

ALLOWED_HOSTS = ['localhost', '127.0.0.1', '134.209.104.166', 'binodverma.com', 'wwww.binodverma.com']

INSTALLED_APPS = [
    'django.contrib.admin',
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'userauth',
    'blog',
    'tool_kit',
    'appointments',
    'contact',
    'adminp',
    'resource_store',
    'implinks',
]

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [BASE_DIR / 'templates'],
        'APP_DIRS': True,
        'OPTIONS': {
            'context_processors': [
                'django.template.context_processors.debug',
                'django.template.context_processors.request',
                'django.contrib.auth.context_processors.auth',
                'django.contrib.messages.context_processors.messages',
            ],
        },
    },
]

MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'django.contrib.sessions.middleware.SessionMiddleware',
    'django.middleware.common.CommonMiddleware',
    'django.middleware.csrf.CsrfViewMiddleware',
    'django.contrib.auth.middleware.AuthenticationMiddleware',
    'django.contrib.messages.middleware.MessageMiddleware',
    'django.middleware.clickjacking.XFrameOptionsMiddleware',
]

ROOT_URLCONF = 'spiderweb.urls'

WSGI_APPLICATION = 'spiderweb.wsgi.application'


import dj_database_url

DATABASES = {
    'default': dj_database_url.config(default='sqlite:///db.sqlite3')
}

BASE_DIR = Path(__file__).resolve().parent.parent
# Static files (CSS, JavaScript, Images)
STATIC_URL = '/static/'

# Static root for production
STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles/')


# Additional directories where Django looks for static file

AUTH_USER_MODEL = 'userauth.CustomUser'

AUTHENTICATION_BACKENDS = [
    'django.contrib.auth.backends.ModelBackend',  # Default

]


AUTH_PASSWORD_VALIDATORS = [
    {
        'NAME': 'django.contrib.auth.password_validation.UserAttributeSimilarityValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.MinimumLengthValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.CommonPasswordValidator',
    },
    {
        'NAME': 'django.contrib.auth.password_validation.NumericPasswordValidator',
    },
]


LANGUAGE_CODE = 'en-us'

TIME_ZONE = 'UTC'

USE_I18N = True

USE_TZ = True


DEFAULT_AUTO_FIELD = 'django.db.models.BigAutoField'

# Media files (Uploaded content)
MEDIA_URL = '/media/'
MEDIA_ROOT = BASE_DIR / 'media'

# Tells Django to use the X-Forwarded-Host header from the proxy, allowing it to know the original host requested by the client.
USE_X_FORWARDED_HOST = True
# Tells Django to use the X-Forwarded-Port header from the proxy, indicating the port number used by the client.
USE_X_FORWARDED_PORT = True
# Instructs Django to trust the X-Forwarded-Proto header, which is set by the proxy server, to determine whether the request is secure (HTTPS).
SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')

# Forces all HTTP requests to be redirected to HTTPS.
SECURE_SSL_REDIRECT = True

# Ensures that the CSRF cookie is only sent over HTTPS connections.
CSRF_COOKIE_SECURE = True
# Ensures that the session cookie is only sent over HTTPS connections.
SESSION_COOKIE_SECURE = True

# Enables HTTP Strict Transport Security (HSTS) for the specified duration (in seconds), forcing browsers to only connect via HTTPS.
SECURE_HSTS_SECONDS = 31536000
# Applies HSTS policy to all subdomains.
SECURE_HSTS_INCLUDE_SUBDOMAINS = True
# Allows the domain to be included in browsers' HSTS preload list, ensuring maximum protection.
SECURE_HSTS_PRELOAD = True

# Enables the X-Content-Type-Options header, preventing browsers from MIME-sniffing a response away from the declared content-type.
SECURE_CONTENT_TYPE_NOSNIFF = True
# Controls the information sent in the Referer header, improving privacy and security by not sending the referrer from HTTPS to HTTP.
SECURE_REFERRER_POLICY = 'no-referrer-when-downgrade'

/etc/nginx/sites-available/spidernet

 Redirect all HTTP traffic to HTTPS
server {
    listen 80;
    server_name 134.209.104.166 binodverma.com www.binodverma.com;

    # Redirect all HTTP to HTTPS
    return 301 https://$host$request_uri;
}

# Main server block for HTTPS
server {
    listen 443 ssl;  # Listen on port 443 for HTTPS
    server_name binodverma.com www.binodverma.com;

    ssl_certificate /etc/letsencrypt/live/binodverma.com/fullchain.pem; # SSL certificate path
    ssl_certificate_key /etc/letsencrypt/live/binodverma.com/privkey.pem; # SSL certificate key path
    include /etc/letsencrypt/options-ssl-nginx.conf; # SSL options for security
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;   # Diffie-Hellman parameter

    # Location block for static files
    location /static/ {
        root /home/vishesh/spidernet/staticfiles;
    }

    # Location block for media files
    location /media/ {
        alias /home/vishesh/spidernet/media/;  # Must be the same as MEDIA_ROOT
        access_log /var/log/nginx/media_access.log;
        error_log /var/log/nginx/media_error.log;
    }

    # Proxying to Gunicorn
    location / {
        include proxy_params;
        proxy_pass http://unix:/run/gunicorn.sock;

        access_log /var/log/nginx/django_access.log;
        error_log /var/log/nginx/django_error.log;
    }
}

# Fallback server block for non-matching hostnames
server {
    listen 80;
    server_name 134.209.104.166 binodverma.com www.binodverma.com;

    return 404; # Return 404 for any unmatched requests
}

/etc/systemd/system/gunicorn.service

[Unit]
Description=gunicorn daemon
Requires=gunicorn.socket
After=network.target

[Service]
User=www-data
Group=www-data
WorkingDirectory=/home/vishesh/spidernet
ExecStart=/home/vishesh/env/bin/gunicorn \
          --access-logfile - \
          --workers 3 \
          --bind unix:/run/gunicorn.sock \
          spiderweb.wsgi:application

[Install]
WantedBy=multi-user.target

(env) vishesh@cactus:~/spidernet$ ls
README.md  adminp  appointments  blog  contact  db.sqlite3  gunicorn_config.py  implinks  manage.py  media  requirements.txt  resource_store  spiderweb  static  staticfiles  templates  test.py  tool_kit  userauth

Side note: Please do not open any additional new topics for your deployment issues here. (See the forum FAQ section “Keep it Tidy”.)

We will try to help you resolve this, but you need to help the process along.

Statements like this do not help us help you. You lose time by making us prompt you for this information. In all cases, when you try something, you need to specify what it is that you have tried by showing the url being requested, any recent file or configuration changes (the actual changes not just a description of what the changes were), and all the details that you can find about the errors themselves - log files, console messages, etc.

To start, I’m going to go back to what I always recommend people do - get static and media files out of your home directory and into more “nginx-friendly” directories.

Suggestions:

  • Change STATIC_ROOT to be of the pattern /var/www/html/<project name>
    i.e., STATIC_ROOT=/var/www/html/spidernet/static/
  • Change MEDIA_ROOT to be something like /var/www/html/spidernet/media/
  • Ensure that /var/www/html exists, is owned by www-data and group www-data, and has the “sticky bit” set on the group.
    • mkdir -p /var/www/html
    • chown www-data:www-data /var/www/html
    • chmod 2775 /var/www/html
      Repeat this for /var/www/html/spidernet, /var/www/html/spidernet/static, /var/www/html/spidernet/media
      (You may need to be root or use sudo for these)
  • Run collectstatic, verifying that each project has deployed the static files to the proper directories in /var/www/html/spidernet/static/.
  • Change your nginx configuration for these static directories.

Restart everything.

Beyond this, we’re going to need to see details of exactly what the current issues are that you are encountering.

1 Like

I apologize for opening multiple topics for this issue. I will keep it in mind for future.
Let’s suppose I have to start deploying the django app on an ubuntu server on digitalocean.
First, what should be the ideal project structure in reference to production.
Secondly I wanted to ask what if some of my CSS are in-line in the template and not stored as an external file in static/css, does that affect the process?
Should I go with certbot for SSL verification?

Will change the STATIC_ROOT and MEDIA_ROOT in new configuration

The project structure itself doesn’t change. The only changes affect those files needing to be served by nginx instead of Django, which is handled by the settings such as STATIC_ROOT and MEDIA_ROOT.

Not at all. Template references do not change based on deployment because Django remains responsible for rendering templates. They are not handled directly by nginx.

That’s a personal choice. I have used it for my personal sites for at least 5 years now without any problems at all. (Might even be longer, I don’t remember for sure.)

1 Like

Previously I had my project in /home/. I read your reply somewhere that we should keep the project in /opt/. However, in doing so do I have to make any other necessary path adjustment? Also what should be the STATIC_URL?

That’s a more subtle issue, but yes, that’s a good general recommendation. In general terms, there are advantages to having your project reside in “neutral” territory, with the idea being that you can have multiple people having administrative access to the project without needing to be root. (e.g., when you have multiple sys-admins needing to deploy or troubleshoot the system) Another option having the same net effect is to create a specific account for each project and deploying the project to the /home directory of its account.
(I’ve spent virtually my entire career where the “principle of least privilege” has been an overriding concern.)

You shouldn’t need to. Pretty much everything in your configuration referencing project-relative directories should either be identified as relative to your BASE_DIR setting or relative to an imported app.

Your STATIC_URL corresponds to the location directive in your nginx configuration. The {% static %} tag defines a “prefix” to the url being generated as a reference. It’s that prefix that nginx is going to match to find which location directive to use to handle that request.

1 Like

After creating all the directories, and giving them permissions, when I run python3 manage.py collectstatic, it is then asking permission for all the specific static files

PermissionError: [Errno 13] Permission denied: '/var/www/html/spidernet/static/css/styles.css'

Again, you may need to be root for this.

The site is running logically fine now on the ip, however its still the half missing css and images.

settings.py

STATIC_URL = '/static/'
STATIC_ROOT = '/var/www/html/spidernet/static/'
STATICFILES_DIRS = [
    BASE_DIR / 'static',
]

MEDIA_URL = '/media/'
MEDIA_ROOT = '/var/www/html/spidernet/media/'

/etc/nginx/sites-enabled/spidernet

 location /static/ {
        root /var/www/spidernet/static;
    }
    location /media/ {
        root /var/www/spidernet/static;
    }

Your STATIC_ROOT and MEDIA_ROOT settings don’t match the path of your root directives for nginx.

2024/09/18 16:23:39 [error] 65259#65259: *11 open() "/var/www/html/spidernet/static/static/images/car2.jpg" failed (2: No such file or directory), client: 106.219.166.128, server: www.binodverma.com, request: "GET /static/images/car2.jpg HTTP/1.1", host: "157.245.51.34", referrer: "http://157.245.51.34/"
2024/09/18 16:23:39 [error] 65259#65259: *13 open() "/var/www/html/spidernet/static/static/images/car3.jpg" failed (2: No such file or directory), client: 106.219.166.128, server: www.binodverma.com, request: "GET /static/images/car3.jpg HTTP/1.1", host: "157.245.51.34", referrer: "http://157.245.51.34/"
2024/09/18 16:23:47 [error] 65259#65259: *9 open() "/var/www/html/spidernet/static/static/images/hero-image.jpg" failed (2: No such file or directory), client: 106.219.166.128, server: www.binodverma.com, request: "GET /static/images/hero-image.jpg HTTP/1.1", host: "157.245.51.34", referrer: "http://157.245.51.34/"
2024/09/18 16:35:31 [error] 65376#65376: *3 open() "/var/www/html/spidernet/static/static/images/hero-image.jpg" failed (2: No such file or directory), client: 106.219.166.128, server: www.binodverma.com, request: "GET /static/images/hero-image.jpg HTTP/1.1", host: "157.245.51.34"
2024/09/18 16:46:23 [error] 65747#65747: *1 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 185.191.126.213, server: www.binodverma.com, request: "GET / HTTP/1.1", upstream: "http://unix:/run/gunicorn.sock:/", host: "157.245.51.34:80"
2024/09/18 16:58:23 [error] 65818#65818: *4 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 157.245.51.34, server: www.binodverma.com, request: "GET / HTTP/1.1", upstream: "http://unix:/run/gunicorn.sock:/", host: "157.245.51.34"
2024/09/18 16:58:30 [error] 65818#65818: *6 connect() to unix:/run/gunicorn.sock failed (111: Connection refused) while connecting to upstream, client: 66.249.66.39, server: www.binodverma.com, request: "GET /robots.txt HTTP/1.1", upstream: "http://unix:/run/gunicorn.sock:/robots.txt", host: "binodverma.com"
2024/09/18 16:58:30 [error] 65818#65818: *8 connect() to unix:/run/gunicorn.sock failed (111: Connection refused) while connecting to upstream, client: 66.249.66.40, server: www.binodverma.com, request: "GET /blog/post/8/ HTTP/1.1", upstream: "http://unix:/run/gunicorn.sock:/blog/post/8/", host: "binodverma.com"
2024/09/18 16:58:47 [error] 65818#65818: *10 connect() to unix:/run/gunicorn.sock failed (111: Connection refused) while connecting to upstream, client: 157.245.51.34, server: www.binodverma.com, request: "GET / HTTP/1.1", upstream: "http://unix:/run/gunicorn.sock:/", host: "157.245.51.34"
2024/09/18 17:07:08 [error] 65818#65818: *12 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 143.110.222.166, server: www.binodverma.com, request: "GET / HTTP/1.1", upstream: "http://unix:/run/gunicorn.sock:/", host: "157.245.51.34"

Argh, I always get root and alias confused with nginx.

Look at the reference in the error message:

See where it has the path component static doubled?

That’s an indication that:

Should be either (your choice):
root /var/www/html/spidernet;
or
alias /var/www/html/spidernet/static/;

Thank you sir @KenWhitesell. Your insights and resources had been extremely helpful for my project. Thank you for having a big heart for helping people, and a big bag of knowledge. My project is up and running without any issues.