GeoDjango Data Transformation Issue in Production (non-localhost only)

Using Django Deployment
1

GeoDjango Data Transformation Issue in Production (non-localhost only)

My code running in

D1. localhost development configuration → fine
D2. localhost production → fine
D3. live ip / fqdn production (xxx.xxx.x.xxx, https://someproject.com) → not ok

D2 and D3 uses the exact production configuration env file, only difference is the browser address. Aka, same AWS cloudfront + S3, same production PostgreSQL etc.

Specs

django=4.2.7
geos=3.12.1
geoip2=4.7.0
gdal=3.8.1
libgdal-arrow-parquet=3.8.1

Data Transformation Issue in D3

CreateView, no wrong map data point, no log error

image
image1249×738 106 KB

For TemplateView, DetailView, no wrong map data point, below log error

TemplateView

image
image1920×714 143 KB

DetailView

image
image1920×698 175 KB

2023-12-12 03:57:51|ERROR|/opt/conda/envs/someproject/lib/python3.12/site-packages/django/contrib/gis/gdal/libgdal.py|err_handler|GDAL_ERROR 1: b'PROJ: proj_create_from_database: Open of /opt/conda/envs/someproject/share/proj failed'
2023-12-12 03:57:51|ERROR|/opt/conda/envs/someproject/lib/python3.12/site-packages/django/contrib/gis/gdal/libgdal.py|err_handler|GDAL_ERROR 1: b'PROJ: proj_create_from_database: Open of /opt/conda/envs/someproject/share/proj failed'

For Admin detail + UpdateView, wrong map data point issue, below log error

Admin detail

image
image1236×691 94.4 KB

UpdateView

image
image1054×733 88 KB

2023-12-12 04:03:34|ERROR|/opt/conda/envs/someproject/lib/python3.12/site-packages/django/contrib/gis/gdal/libgdal.py|err_handler|GDAL_ERROR 1: b'PROJ: proj_create_from_database: Open of /opt/conda/envs/someproject/share/proj failed'
2023-12-12 04:03:34|ERROR|/opt/conda/envs/someproject/lib/python3.12/site-packages/django/contrib/gis/gdal/libgdal.py|err_handler|GDAL_ERROR 1: b'PROJ: proj_create_from_database: Open of /opt/conda/envs/someproject/share/proj failed'
2023-12-12 04:03:34|ERROR|/opt/conda/envs/someproject/lib/python3.12/site-packages/django/contrib/gis/gdal/libgdal.py|err_handler|GDAL_ERROR 1: b'PROJ: proj_create_from_database: Open of /opt/conda/envs/someproject/share/proj failed'
2023-12-12 04:03:34|ERROR|/opt/conda/envs/someproject/lib/python3.12/site-packages/django/contrib/gis/gdal/libgdal.py|err_handler|GDAL_ERROR 1: b'PROJ: proj_create_from_database: Open of /opt/conda/envs/someproject/share/proj failed'
2023-12-12 04:03:34|ERROR|/opt/conda/envs/someproject/lib/python3.12/site-packages/django/contrib/gis/forms/widgets.py|get_context|Error transforming geometry from srid '4326' to srid '3857' (OGR failure.)
2023-12-12 04:03:34|ERROR|/opt/conda/envs/someproject/lib/python3.12/site-packages/django/contrib/gis/forms/widgets.py|get_context|Error transforming geometry from srid '4326' to srid '3857' (OGR failure.)
2023-12-12 04:03:34|ERROR|/opt/conda/envs/someproject/lib/python3.12/site-packages/django/contrib/gis/gdal/libgdal.py|err_handler|GDAL_ERROR 1: b'PROJ: proj_create_from_database: Open of /opt/conda/envs/someproject/share/proj failed'
2023-12-12 04:03:34|ERROR|/opt/conda/envs/someproject/lib/python3.12/site-packages/django/contrib/gis/gdal/libgdal.py|err_handler|GDAL_ERROR 1: b'PROJ: proj_create_from_database: Open of /opt/conda/envs/someproject/share/proj failed'

Thoughts

Since D2 and D3 uses the exact production configuration env file. I think

  1. Cloudfront, S3, PostgreSQL is not the issue
  2. Issue maybe at reverse proxy, or django level

The OGR failure seems to suggest it’s database error, but based on D2, I think PostgreSQL is not the issue.

Any idea?

2

I figured it out …

For my D2, I activate my project-specific conda environment in my linux VM and runserver.

For my D3, it’s deployed as docker image. I run it without activating that project-specific conda env. Thus the proj library path is not activated / defined.

The solution is to activate that project-specific env before running server.

Rookie mistake :man_facepalming:

3

Just to add an observation here, and for the benefit of anyone else reading this thread.

From the docs for runserver

DO NOT USE THIS SERVER IN A PRODUCTION SETTING. It has not gone through security audits or performance tests.

[emphasis added]

If this is going to be a system that other people are going to use, do not do this for your production deployment. Use a real wsgi server like gunicorn or uwsgi.

Otherwise, you’ll be making another

4

Great point, I did run using gunicorn instead of runserver for production. Fix that part :slight_smile:

For people who use conda environment, and just moved from VM to container, this might be something to look out for.