Django collectstatic doesn't work without a db

I’m using couchdb for a django app. When the couchdb is down i’m unable to do python manage.py collectstatic. Is there anyway to run collectstatic with db being down. In local env i can bring the db up and do the operation. But here i’m dockerizing the django app which will be connected to a couchdb container and couch will be run after build of both django and couchdb image is completed.

I also have a dockerized setup.
I just started only the Django app without the PostgresSQL db associated with it and I was able to run the collectstatic command without any problem.

What is the error you’re getting? You can run collectstatic -v 3 for more verbosity.

  File "/code/webapp/webappmain/models.py", line 7, in <module>
    db = couch[settings.TABLE]
  File "/usr/local/lib/python3.6/site-packages/couchdb/client.py", line 145, in __getitem__
    db.resource.head() # actually make a request to the database
  File "/usr/local/lib/python3.6/site-packages/couchdb/http.py", line 557, in head
    return self._request('HEAD', path, headers=headers, **params)
  File "/usr/local/lib/python3.6/site-packages/couchdb/http.py", line 592, in _request
    credentials=self.credentials)
  File "/usr/local/lib/python3.6/site-packages/couchdb/http.py", line 295, in request
    conn = self.connection_pool.get(url)
  File "/usr/local/lib/python3.6/site-packages/couchdb/http.py", line 515, in get
    conn.connect()
  File "/usr/local/lib/python3.6/http/client.py", line 952, in connect
    (self.host,self.port), self.timeout, self.source_address)
  File "/usr/local/lib/python3.6/socket.py", line 704, in create_connection
    for res in getaddrinfo(host, port, 0, SOCK_STREAM):
  File "/usr/local/lib/python3.6/socket.py", line 745, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -2] Name or service not known

This is the error i get. Same reflects when i do python manage.py collectstatic in my local also by removing db backend from settings

Ok - it looks like you might have a direct reference to your setting from within your models.py file (see the first entry of the log).

You’ve got a reference to settings.TABLE there, which if that is what your DB-related settings are, explains why you’re getting the error message.

If you need to run this without having the definition within the settings file, you need to put some sort of “guard” around that reference. (I’d need to see your models.py file to make any more specific suggestion than that.

@KenWhitesell, I couldn’t find any couchdb engines to use in settings database. So i commented out Database in settings.py and written a custom connection in model.

from django.conf import settings
couch = couchdb.Server(settings.DB_URL)
db = couch[settings.TABLE]

db url and table are fetched from .env file. the issue here is when i dockerize the application db container is built but it is not running. Similarly in localhost if my couchdb is not up, the above mentioned error is thrown. May be bcs manage.py checks whether database is available or not. Is there anything i can do to overcome this validation as i’m not using django’s default sqlite3 db as well

To run collectstatic, we need to give the command python manage.py collectstaic right. But if i do only python manage.py . Same error is thrown,may be bcs django validates if db exists or not.I dont understand why manage.py is dependent on db

The manage command is not dependent on the database. The error you posted points directly at this section of code:

Since this code is at the module level and not within a function or class, it’s going to be executed when the file is imported, and manage.py will import your models.

You need to check to see what couchdb.Server(settings.DB_URL) is returning when your database isn’t up. Whatever it’s returning is causing the next line to throw an error.

You need to put some guards in or around that block of code so that it doesn’t throw an error if the database isn’t available.

(Note: Some of this may still be conjecture because it is not clear whether or not what you posted is the complete stack trace.)

@KenWhitesell, Here is the full stacktrace when i’m running my app in local without the db being up.

(venv) C:\DartApp\webapp>python manage.py collectstatic
Traceback (most recent call last):
  File "manage.py", line 22, in <module>
main()
  File "manage.py", line 18, in main
execute_from_command_line(sys.argv)
  File "C:\DartApp\venv\lib\site-packages\django\core\management\__init__.py", line 401, in execute_from_command_line
utility.execute()
  File "C:\DartApp\venv\lib\site-packages\django\core\management\__init__.py", line 377, in execute
django.setup()
  File "C:\DartApp\venv\lib\site-packages\django\__init__.py", line 24, in setup
apps.populate(settings.INSTALLED_APPS)
  File "C:\DartApp\venv\lib\site-packages\django\apps\registry.py", line 114, in populate
app_config.import_models()
  File "C:\DartApp\venv\lib\site-packages\django\apps\config.py", line 211, in import_models
self.models_module = import_module(models_module_name)
  File "C:\Users\BharathK\AppData\Local\Programs\Python\Python36\lib\importlib\__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "C:\DartApp\webapp\webappmain\models.py", line 7, in <module>
db = couch[settings.TABLE]
  File "C:\DartApp\venv\lib\site-packages\couchdb\client.py", line 145, in __getitem__
db.resource.head() # actually make a request to the database
  File "C:\DartApp\venv\lib\site-packages\couchdb\http.py", line 557, in head
return self._request('HEAD', path, headers=headers, **params)
  File "C:\DartApp\venv\lib\site-packages\couchdb\http.py", line 592, in _request
credentials=self.credentials)
  File "C:\DartApp\venv\lib\site-packages\couchdb\http.py", line 295, in request
conn = self.connection_pool.get(url)
  File "C:\DartApp\venv\lib\site-packages\couchdb\http.py", line 515, in get
conn.connect()
  File "C:\Users\BharathK\AppData\Local\Programs\Python\Python36\lib\http\client.py", line 936, in connect
(self.host,self.port), self.timeout, self.source_address)
  File "C:\Users\BharathK\AppData\Local\Programs\Python\Python36\lib\socket.py", line 704, in create_connection
for res in getaddrinfo(host, port, 0, SOCK_STREAM):
  File "C:\Users\BharathK\AppData\Local\Programs\Python\Python36\lib\socket.py", line 745, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno 11001] getaddrinfo failed

If i give only python manage.py with no db running, same error i receive.

(venv) C:\DartApp\webapp>python manage.py
Traceback (most recent call last):
  File "manage.py", line 22, in <module>
    main()
  File "manage.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "C:\DartApp\venv\lib\site-packages\django\core\management\__init__.py", line 401, in execute_from_command_line
    utility.execute()
  File "C:\DartApp\venv\lib\site-packages\django\core\management\__init__.py", line 377, in execute
    django.setup()
  File "C:\DartApp\venv\lib\site-packages\django\__init__.py", line 24, in setup
    apps.populate(settings.INSTALLED_APPS)
  File "C:\DartApp\venv\lib\site-packages\django\apps\registry.py", line 114, in populate
    app_config.import_models()
  File "C:\DartApp\venv\lib\site-packages\django\apps\config.py", line 211, in import_models
    self.models_module = import_module(models_module_name)
  File "C:\Users\BharathK\AppData\Local\Programs\Python\Python36\lib\importlib\__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "C:\DartApp\webapp\webappmain\models.py", line 7, in <module>
    db = couch[settings.TABLE]
  File "C:\DartApp\venv\lib\site-packages\couchdb\client.py", line 145, in __getitem__
    db.resource.head() # actually make a request to the database
  File "C:\DartApp\venv\lib\site-packages\couchdb\http.py", line 557, in head
    return self._request('HEAD', path, headers=headers, **params)
  File "C:\DartApp\venv\lib\site-packages\couchdb\http.py", line 592, in _request
    credentials=self.credentials)
  File "C:\DartApp\venv\lib\site-packages\couchdb\http.py", line 295, in request
    conn = self.connection_pool.get(url)
  File "C:\DartApp\venv\lib\site-packages\couchdb\http.py", line 515, in get
    conn.connect()
  File "C:\Users\BharathK\AppData\Local\Programs\Python\Python36\lib\http\client.py", line 936, in connect
    (self.host,self.port), self.timeout, self.source_address)
  File "C:\Users\BharathK\AppData\Local\Programs\Python\Python36\lib\socket.py", line 704, in create_connection
    for res in getaddrinfo(host, port, 0, SOCK_STREAM):
  File "C:\Users\BharathK\AppData\Local\Programs\Python\Python36\lib\socket.py", line 745, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno 11001] getaddrinfo failed

Ok, so this does more confirm what I had thought.

You need to check to see what couchdb.Server(settings.DB_URL) is returning when your database isn’t up or isn’t configured.

You need to put some guards in or around that block of code so that it doesn’t throw an error if the database isn’t available.

@KenWhitesell, For dockerizing i have added an initializer which will create the table once the db is up i.e,( build is done and container starts). The actual issue here is i need to copy the staticfiles to nginx image to serve. This process is happening during the build time, so db container wont be up until then. The error thrown will be handled once the container has started.

So here i dont want to handle the error bcs it will be handled when initializer i added starts. Here i want to collect my static files and copy it to ngnix image directory during build time where none of the containers will be running(including couchdb). I can see python manage.pydepends on model and couldn’t find the db instance. Is it really so that manage.py requires a db running to operate?

If i start my db and do python manage.py it works like charm. So is it common django behavior that manage.py requires a db to execute?

Manage.py does not require a db to execute.

You have written code that requires a db to execute.

1 Like

Ohh i see…

Are you aware of any engine for couchdb which i can use here. In my case i have commented the database section in settings.py and added the db instance in my model.May be that is the issue.

settings.py

# DATABASES = {
#     'default': {
#         'ENGINE': 'django.db.backends.sqlite3',
#         'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
#     }
# }

models.py

import couchdb
from django.conf import settings
couch = couchdb.Server(settings.DB_URL)
db = couch[settings.TABLE]
doc={"name":"test","value":1}  
db["test1"]=doc     #just a save operation for demo 

Here in models.py i have written normal python couchdb operations to save, retrieve and updated data

I’m not specifically aware of a couchdb engine. You can check pypi.org and/or djangopackages.org to see if there’s anything there that would work for you.

But in the absence of that:

I’ll add at this point that the initialization code you’ve shown in your models.py file probably would more appropriately be placed in your settings file. You can then access it from any models, views and utility modules you may create.

Thanks a lot KenWhitesell for your time. I have kept the db initialization in try except block and it seems to work fine.
But one doubt though what difference would it make if i write the db initialization in settings or models?
I have written db initialization in models bcs all the operations related to db are written there so didn’t want to import it from settings. It wouldn’t be a problem right?

Is it a problem? No.

It’s purely a subjective style issue.

There could be practical implications if you find yourself using that connection across multiple Django apps, where you want to use that connection in more than one models.py file. But as long as that doesn’t happen, no, it’s no problem at all.

1 Like

Thank you Ken. Now i’m clear with it :blush: