Ticket #34746 - Use reprlib instead of pprint to avoid CPU/memory spikes in the event of a 5xx exception

Django Internals Mentorship
1

Hi,

I want to work on Ticket #34746 and wanted some feedback about the approach which has been suggested in the ticket itself.

Currently, in the event of an exception inside a view, to display the local variables for a frame in a stack trace, we use pprint. This causes the CPU and memory usage to spike if the variable in question is large in size. This data is trimmed down later, but not before being fully rendered.

A possible solution is to use reprlib instead of pprint with appropriate limits to control the rendering, although losing the pretty printing aspect. Is this solution feasible? Are there any other ways this can be achieved?

Current code:

frames = self.get_traceback_frames()
for i, frame in enumerate(frames):
    if "vars" in frame:
        frame_vars = []
        for k, v in frame["vars"]:
            v = pprint(v)
            # Trim large blobs of data
            if len(v) > 4096:
                v = "%s… <trimmed %d bytes string>" % (v[0:4096], len(v))
            frame_vars.append((k, v))
        frame["vars"] = frame_vars
    frames[i] = frame

Trimming of the variables, I believe was introduced as a fix to Ticket #20368, in order to not run into MemoryErrors which would prevent rendering of the traceback page.

2

I just replied in this thread where another developer, @keerthivasansa , is also planning on working on the ticket: Usage of reprlib instead of pprint in exception reporting . Perhaps you can coordinate and join forces to work on this ticket?

3

Hi Adam, thanks for replying. It seems a PR has already been raised and the review is in progress. I’m not sure of the work left, but I’ll be happy to help :slight_smile:

4

Hey Piyush, appreciate the efforts. I had already completed relevant patches and waiting only for a review from a maintainer. You can look at my PR and if some improvements are requested by the team, we can make those together. Thanks!

Fix #34746: Add a size limit for variable size when pretty printing exception report by keerthivasansa · Pull Request #17809 · django/django (github.com)