Is it feasible to improve streaming HTML in Django?

Hello, Djangonauts!

I’ve been excited to leverage streaming HTML in Django since I read the impressive series of blog posts from Taylor Hunt, where he detailed how he vastly improved the experience of a Fortune 20 website, even on slow Android phones by using as little JavaScript as he could get away with and leveraging the old technology of streaming HTML.

I created a small proof of concept of streaming HTML in Django. It renders a home page of an e-commerce website that acts like it recommends specific items for the current viewer. I added an asyncio.sleep call to mimic a slow database query or API call. Thanks to the streaming HTML, all of the page before that section renders quickly, giving the impression of a performant website.

I’m excited that this PoC shows it’s possible, but I’m a little disappointed with how I got it to work. I had to:

  • render the shell, or base, HTML file that holds all the header content
  • split it apart at a place where I knew I could insert the content
  • create two additional templates to handle the parts of the page before and after the slow content
  • then render a fourth template for the slow data

An excerpt of the file is below:

customized_recommendations = [
    # example data
    dict(name='Comfy Chair', discount_price=745.00, normal_price=800.00, review_count=1550, img='product1.jpg'),

async def recommendations():
    # split the shell template to sandwich the rest of the HTML
    pre_shell, post_shell = render_to_string('shell.html').split('<!-- footer -->')
    yield pre_shell
    yield render_to_string('home/home_pre.html')
    for item in customized_recommendations:
        await asyncio.sleep(.7)  # Faking an expensive database query or slow API
        yield render_to_string('home/_item.html', dict(recommendation=item))
    yield render_to_string('home/home_post.html')
    yield post_shell

async def index(request):
    return StreamingHttpResponse(recommendations())

Compare that to a similar implementation in Starlette / Jinja, where I could use one template:

async def homepage(request):
    recommendations = [
        dict(name='Comfy Chair', discount_price=745.00, normal_price=800.00, review_count=1550, img='product1.jpg'),

    async def slow_recommendations():
        # fake a slow query or API request
        for r in recommendations:
            delay = random.randint(0, 500) / 100
            await asyncio.sleep(delay)
            yield r

    template = templates.get_template('index.html')
    return StreamingResponse(

Calton asked me to look into what Jinja is doing with that generate_async method. I think it uses an async event loop to iterate through the templates, sending chunks down the wire when they’re ready and looping in a holding pattern while the async data call resolves.

I don’t know if improving the developer experience in this way is feasible, but I call it to your attention to see what you think.

Thank you so much for all your work at making such an incredible platform!


This has always been one of my “white whales” with async - where we can finally close the loop on rendering, as it were.

As it stands, I think we’d have to go in and overhaul the template engine to accomplish this; async functions, iterators and generators are different enough that they can’t really be slotted in, and the way templates render is especially difficult to work with in this context.

I’ve not been in that codebase in a while, but the first step would be having someone who knows templating somewhat well go and learn it and establish if there’s a core node render loop we can async-iterate over in a similar way to Jinja.


I wouldn’t panic, it’s hasn’t changed :stuck_out_tongue_winking_eye:

(Sorry for the noise, couldn’t resist. :slight_smile: )

1 Like