How do you serve charts over large time-series data in Django?

Using Django
1

Hi all,

At work I maintain a Django backend that handles high-frequency sensor data. Charting long time ranges was a real problem for us: pulling millions of rows to draw a line that is ~1200px wide is slow, heavy on memory, and pointless, since the screen can’t show more than a few thousand points anyway.

We ended up writing our own downsampling and pre-aggregation layer. It works, but it took real effort to get right (late data, memory limits, keeping peaks visible), and I suspect many teams are writing the same thing from scratch.

So I’m curious how others handle this:

  1. Where does your time-series data live? Plain Postgres, TimescaleDB, Parquet files + DuckDB, something else?
  2. When a user asks for a chart over weeks or months, what do you do? Return all rows, aggregate per hour/day, downsample (LTTB, M4…), or something custom?
  3. If you’re on a managed Postgres without TimescaleDB (RDS, Cloud SQL…), how did you solve it?
  4. What was the most painful part?

I’m considering building an open-source package for this (efficient downsampling + rollups + automatic resolution selection, for both Postgres and Parquet/DuckDB), but first I want to understand whether this is a common pain or just ours.

Thanks!

2

i did some work with time-series data a while ago. It was basically a collection of snmp data from devices being tossed into a graphite DB. For our purposes, the built-in grafana interface / dashboards were more than adaquate for our needs.
(In particular, having it pre-calculate the day/week/month/year averages made producing those types of summaries very fast.)

The most painful part was defining the granularity and ensuring that the data was collected in the most timely fashion.

3

Thanks, Ken, that’s really helpful. Graphite + Grafana with pre-computed rollups sounds like a clean setup, and it’s a good reminder that a lot of teams don’t serve these charts through Django at all. Defining the granularity up front is a great point too. Appreciate you sharing it.