I created a small proof of concept of streaming HTML in Django. It renders a home page of an e-commerce website that acts like it recommends specific items for the current viewer. I added an
asyncio.sleep call to mimic a slow database query or API call. Thanks to the streaming HTML, all of the page before that section renders quickly, giving the impression of a performant website.
I’m excited that this PoC shows it’s possible, but I’m a little disappointed with how I got it to work. I had to:
- render the shell, or base, HTML file that holds all the header content
- split it apart at a place where I knew I could insert the content
- create two additional templates to handle the parts of the page before and after the slow content
- then render a fourth template for the slow data
An excerpt of the
views.py file is below:
customized_recommendations = [ # example data dict(name='Comfy Chair', discount_price=745.00, normal_price=800.00, review_count=1550, img='product1.jpg'), ... ] async def recommendations(): # split the shell template to sandwich the rest of the HTML pre_shell, post_shell = render_to_string('shell.html').split('<!-- footer -->') yield pre_shell yield render_to_string('home/home_pre.html') for item in customized_recommendations: await asyncio.sleep(.7) # Faking an expensive database query or slow API yield render_to_string('home/_item.html', dict(recommendation=item)) yield render_to_string('home/home_post.html') yield post_shell async def index(request): return StreamingHttpResponse(recommendations())
Compare that to a similar implementation in Starlette / Jinja, where I could use one template:
async def homepage(request): recommendations = [ dict(name='Comfy Chair', discount_price=745.00, normal_price=800.00, review_count=1550, img='product1.jpg'), ... ] async def slow_recommendations(): # fake a slow query or API request for r in recommendations: delay = random.randint(0, 500) / 100 await asyncio.sleep(delay) yield r template = templates.get_template('index.html') return StreamingResponse( content=template.generate_async( request=request, recommendations=slow_recommendations(), ), media_type='text/html', )
Calton asked me to look into what Jinja is doing with that
generate_async method. I think it uses an async event loop to iterate through the templates, sending chunks down the wire when they’re ready and looping in a holding pattern while the async data call resolves.
I don’t know if improving the developer experience in this way is feasible, but I call it to your attention to see what you think.
Thank you so much for all your work at making such an incredible platform!