Cannot convert Unix or UTC times in other arbitrary timezones

[venteto]

I’ve been struggling with this, and not sure what to do at all. I’ve been avoiding using pytz and astimezone as I understand both of these are deprecated. I’m using Django 5.2 and Python 3.13.

I look in my test sqlite database and I’m surprised to not see any timezone offset stored in the field, yet if I attempt to use make_aware in my model method, I get an error indicating that the field is not naive. I don’t know if it is just my settings file that is making the field “aware” rather than any actual timezone info being stored in the the database field itself.

This is the last iteration of my settings and models files that I have used, and my method to try to convert either the Unix or UTC times into New York time either just passes through the UTC time unmodified, or it does yield a time offset by 5 hours but in the wrong direction, into the next day, not earlier in the same day as you would expect New York to be, relative to UTC.

I believe I also tried some of this just directly in a Python REPL without any Django bits, and the New York time was returned as you would expect to be, offset 5 hours from UTC.

# settings.py
USE_TZ = True
TIME_ZONE = 'UTC'

# models.py
from django.db import models
from datetime import datetime, timezone
from zoneinfo import ZoneInfo

class TestTime(models.Model):
    ts_unix = models.PositiveSmallIntegerField(null=True,blank=True)
    ts_utc = models.DateTimeField(null=True,blank=True)
    ts_ny = models.DateTimeField(null=True,blank=True)

    class Meta:
        ordering = ['ts_unix']

    def __str__(self):
        return str(self.ts_unix)

    def get_utc(self):
        unix = self.ts_unix
        utc = datetime.fromtimestamp(unix, tz=timezone.utc)
        try:
            return utc
        except (TypeError, ValueError):
            return None

    def get_ny(self):
        unix = self.ts_unix
        zone = ZoneInfo("America/New_York")
        ts_utc = datetime.fromtimestamp(unix, tz=timezone.utc)

        # ts_ny = ts_utc.astimezone(zone)
        # ts_ny = ts_utc.astimezone(ZoneInfo('America/New_York'))
        # ts_ny = datetime.fromtimestamp(unix, tz=zone)
        ts_ny = ts_utc.replace(tzinfo=ZoneInfo("America/New_York"))
        try:
            return ts_ny
        except (TypeError, ValueError):
            return None

    def save(self, *args, **kwargs):
        self.ts_utc = self.get_utc()
        super(TestTime, self).save(*args, **kwargs)

If, for example, I try to get the New York datetime directly from the Unix timestamp, in the Django model method it only returns a UTC datetime.

# models.py
import datetime as dtm
from django.db import models
from zoneinfo import ZoneInfo

class TestTime(models.Model):
    ts_unix = models.PositiveIntegerField(null=True, blank=True)
    ts_utc = models.DateTimeField(null=True, blank=True)
    ts_ny = models.DateTimeField(null=True, blank=True)
    ...

    def get_ny_time(self):
        try:
            # Convert to New York time using ZoneInfo
            ny_zone = ZoneInfo("America/New_York")
            return dtm.datetime.fromtimestamp(self.ts_unix, tz=ny_zone)
        except Exception as e:
            print(f"Error converting to NY time: {e}")
            return None

But if I move the same logic into a raw Python script, it returns the correct datetime, so it seems like something in Django is causing the problem:

# script.py
import datetime as dtm
from zoneinfo import ZoneInfo

ts_unix = 1741035602

def get_ny_time():
   
    try:
        # Convert to New York time using ZoneInfo
        ny_zone = ZoneInfo("America/New_York")
        return dtm.datetime.fromtimestamp(ts_unix, tz=ny_zone)

    except Exception as e:
        print(f"Error converting to NY time: {e}")
        return None

print(get_ny_time())

❯ python -V
Python 3.13.1
❯ python script.py
2025-03-03 16:00:02-05:00

[anefta]

:

1. Your get_utc() method converts from Unix timestamp to UTC, but you’re not implementing any method to convert to New York time

2. With USE_TZ = True, Django automatically makes datetime fields timezone-aware (using UTC internally), which explains why make_aware showed the field was already aware.

Try this:

# settings.py
USE_TZ = True
TIME_ZONE = 'UTC'

# models.py
from django.db import models
from datetime import datetime
from zoneinfo import ZoneInfo

class TestTime(models.Model):
    # PositiveIntegerField instead of PositiveSmallIntegerField for Unix timestamps
    ts_unix = models.PositiveIntegerField(null=True, blank=True)
    ts_utc = models.DateTimeField(null=True, blank=True)
    ts_ny = models.DateTimeField(null=True, blank=True)

    class Meta:
        ordering = ['ts_unix']

    def __str__(self):
        return str(self.ts_unix)

    def get_utc(self):
        """Convert Unix timestamp to UTC datetime."""
        if self.ts_unix is None:
            return None
        
        try:
            # Create a timezone-aware UTC datetime from Unix timestamp
            return datetime.fromtimestamp(self.ts_unix, tz=datetime.timezone.utc)
        except (TypeError, ValueError):
            return None

    def get_ny_time(self):
        """Convert UTC datetime to New York time."""
        # Get UTC time from either ts_utc field or by converting ts_unix
        utc_time = self.ts_utc if self.ts_utc is not None else self.get_utc()
        if utc_time is None:
            return None
        
        try:
            # Convert to New York time using ZoneInfo
            ny_zone = ZoneInfo("America/New_York")
            return utc_time.astimezone(ny_zone)
        except Exception as e:
            print(f"Error converting to NY time: {e}")
            return None

    def save(self, *args, **kwargs):
        # Auto-populate ts_utc from ts_unix if ts_unix exists and ts_utc doesn't
        if self.ts_unix is not None and self.ts_utc is None:
            self.ts_utc = self.get_utc()
        
        # Auto-populate ts_ny from either ts_utc or converted ts_unix
        # Only if ts_ny is currently None
        if self.ts_ny is None and (self.ts_utc is not None or self.ts_unix is not None):
            self.ts_ny = self.get_ny_time()
        
        super().save(*args, **kwargs)

[venteto]

@anefta

but you’re not implementing any method to convert to New York time

Not sure if you scrolled down to see my def get_ny(self): line, but I amended my model to match your changes. However, the value returned by the get_ny_time method is equal to the value returned by the get_utc method, rather than being five hours earlier.

Also, I was getting the following error from your code …
type object 'datetime.datetime' has no attribute 'timezone'

… until I made these tweaks:

# from datetime import datetime
import datetime

...

    """ in your get_utc method """
            # return datetime.fromtimestamp(self.ts_unix, tz=datetime.timezone.utc)
            return datetime.datetime.fromtimestamp(self.ts_unix, tz=datetime.timezone.utc)

[anefta]

@venteto

Not sure if you scrolled down to see my def get_ny(self): line, but I amended my model to match your changes. However, the value returned by the get_ny_time method is equal to the value returned by the get_utc method, rather than being five hours earlier.

Omg… You are right! I did not scroll down… Hahaha… Sorry about this. I will check this later!

[anefta]

So, if I am not wrong, Django stores datetimes in the database without timezone information (as UTC). When Django retrieves these values, it attaches timezone information based on your settings. This is why the database shows no timezone offset, but Django treats the field as timezone-aware.

You could create a datetime with the correct initial timezone (UTC in your case) and then use astimezone() to convert to the target timezone.

For example:

from django.db import models
from datetime import datetime
from zoneinfo import ZoneInfo

class TestTime(models.Model):
    ts_unix = models.PositiveIntegerField(null=True, blank=True)
    ts_utc = models.DateTimeField(null=True, blank=True)
    ts_ny = models.DateTimeField(null=True, blank=True)
    
    class Meta:
        ordering = ['ts_unix']
    
    def __str__(self):
        return str(self.ts_unix)
    
    def get_utc(self):
        """Convert Unix timestamp to UTC datetime"""
        if not self.ts_unix:
            return None
        
        try:
            # Create a timezone-aware UTC datetime from the Unix timestamp
            return datetime.fromtimestamp(self.ts_unix, tz=ZoneInfo("UTC"))
        except (TypeError, ValueError) as e:
            print(f"Error converting to UTC: {e}")
            return None
    
    def get_ny(self):
        """Convert Unix timestamp to New York datetime"""
        if not self.ts_unix:
            return None
        
        try:
            # First get UTC time
            utc_time = datetime.fromtimestamp(self.ts_unix, tz=ZoneInfo("UTC"))
            
            # Then convert to New York time
            ny_time = utc_time.astimezone(ZoneInfo("America/New_York"))
            
            return ny_time
        except (TypeError, ValueError) as e:
            print(f"Error converting to NY time: {e}")
            return None
    
    def save(self, *args, **kwargs):
        # Set UTC and NY times before saving
        self.ts_utc = self.get_utc()
        self.ts_ny = self.get_ny()
        
        super(TestTime, self).save(*args, **kwargs)

Let me know how it goes

[venteto]

Thanks. I keep getting the same results, only UTC will show up in the actual database and in the admin interface, even if I make the NY model method a column in the admin change list (prior to being called by the save method), as UTC is what is defined in my settings.py.

This is in spite of myriad Google results including Stackoverflow posts indicating all of these varied approaches should work, but unless I’m mistaken Django models seem inherently opposed to handling multiple timezones.

So it seems the only way I can probably solve this is to convert timestamps in a view. I’ve done that and it works.

However, when I add a timezone field to my User model, and add middleware to activate user timezones in templates, that works nicely in user-facing templates, but also overrides the UTC display in my admin interface (unless, I suppose, I give my admin account a timezone of UTC instead of my real timezone, which defeats the purpose of having user timezones).

So it seems maybe I need to dispense with the middleware and do ALL time conversions directly in the view itself, unless there is a better and more idiomatic approach I have not yet encountered (?). Then the ts_utc field in the model and related model methods become superfluous.

My starting point view:

from datetime import datetime as dtm
from zoneinfo import ZoneInfo as zi
from django.views.generic import ListView 
from .models import TestTime

class TestTimeList(ListView):
    model = TestTime
    template_name = 'tt.html'
    
    def get_context_data(self, **kwargs):
        context = super().get_context_data(**kwargs)
        for obj in context['object_list']:
            dt = dtm.fromtimestamp(obj.ts_unix, tz=zi(obj.exchange.tz))
        context['xch_tz'] = dt.strftime("%Y-%m-%d %H:%M:%S %z %Z")
        return context

My starting point template (with the user timezones middleware active):

<tbody>
    {% load tz %}
    {% for object in object_list %}
        <tr>
            <td>{{ object.ts_unix }}</td>
            <td>{{ object.ts_utc|utc|date:"Y-m-d H:i:s e" }}</td>
            <td>{{ xch_tz }}</td>
            {% if user.is_authenticated %}
                <td>
                    {{ object.ts_utc|localtime|date:"Y-m-d H:i:s e" }}
                </td>
            {% endif %}
        </tr>
    {% endfor %}
</tbody>

[KenWhitesell]

@venteto

but unless I’m mistaken Django models seem inherently opposed to handling multiple timezones.

Quoting directly from the docs at Time zones | Django documentation | Django

When support for time zones is enabled, Django stores datetime information in UTC in the database, uses time-zone-aware datetime objects internally, and translates them to the end user’s time zone in templates and forms.

So yes, all data is stored in the database in UTC. How they’re stored internally may vary. But rendering the data in the desired time zone is work done outside the model.

In the general case you should assume that all internal timestamps would be UTC and that the only need for a non-utc date would be when serializing or rendering those values.