It has been bothering me for a while now,reading files is inefficient and many django projects including the one I am working on at the moment, involves multiple partials that are used in multiple other page templates, which contain lots of {% include %} and {% extends %}
On average, per my project, four HTML files are opened and read to render one page.
Is there a way to pre compile all these templates beforehand to avoid all these overhead before deployment. Just like we have python manage.py collectstatic wouldn’t it be nice to have python manage.py compiletemplates?
Actually, on a modern system with any reasonable amount of activity, this isn’t an accurate statement. The operating system caches file data, and if the data is frequently used, will remain in the file buffers, avoiding any physical disk I/O.
Compile into what form? The templates are already HTML, with the tags to render the page-specific content. But the intent of the template is that it finishes rendering based upon the content of the context - which you don’t know until the time that the view renders it.
Granted, you may have some parts of a page that remain constant and don’t need to go through the parsing & rendering process for every page. For those types of issues, Django already provides a caching framework that allows you to cache page fragments.
But, the bottom line here is that before you worry about any of this, you want to definitively prove that rendering your templates is the source of a significant portion of your response time, and that there’s a noticeable improvement to be made by expending any effort on optimizing it, rather than looking at other components within your system.
@KenWhitesell is right that you should benchmark your problems before declaring them as such.
That said, Django already knows that parsing the templates can be slow. (This is not the reading from disk, but the processing to turn them into in-memory data structures for rendering.) As long as you set DEBUG = False on your deployment (which you definitely need to do to secure your site), Django will use the cached template loader and only parse each template file once.
Template files are re-read each time with DEBUG = True in order to allow you to see your changes instantly.
1 Like
By compiling templates, I mean expanding all extends, include blocks of a template file.
For example, if a template view.html extends base.html and includes _card.html etc.
Load and replace all blocks in base.html with the contents of view.html and replace the {% include %} statement with the contents of _card.html. thus creating a new view.html that can be rendered without reading any of the other files.
This I understand does not offer much significant improvement for projects with fewer template files. But if a project has lots of templates files, which in turn contain lots of these two, includes and extends probably with multiple inheritance, I believe that can be an optimization.
With this I believe the problem is solved even though I cannot say to what extend. Maybe unless these files are too large they starve memory, in case the deployment machine has little of them, which is unlikely.
Django templates do not work like this. Extends, include etc are done dynamically on render since the templates they refer to can be in variables.
Therefore the inclining you’re proposing is not, in the general case, possible.
Most users looking for template rendering speed switch to Jinja2. Its compile step effectively converts templates to python functions, allowing some extra optimisations.
1 Like
If you’re interested in templates internals I suggest you check out Armin Ronacher’s DUTH talk which covers how Jinja2 improved on the design of Django Templates: https://youtu.be/rHmljD-oZrY
Hi,
I’m testing and answering via email to see how it works (sorry if it
doesn’t look good?)
Yes it works, but it seems to only have the start of your reply! 