Allow newlines inside {% %} tags

Django Internals
1

After a discussion on mastodon: Jeff Triplett: "@boxed @anze3db@fosstodon.org Agreed. I wish we c…" - Mastodon

That newlines inside {% %} is not allowed makes some formatting of templates more ugly and bad for git blame. It also causes problems for beginners, as the error message when you do this is not great.

I suggest allowing newlines.

tag_re = re.compile(r"({%.*?%}|{{.*?}}|{#.*?#})")

could maybe be changed to

tag_re = re.compile(r"({%.*?%}|{{.*?}}|{#.*?#})", re.DOTALL)

There is a backwards compatibility issue in that

{% 
%}

currently renders {% %} as a string in the template, while it would be an error with this change.

Maybe this needs to be an option on the template settings, with a deprecation warning if you haven’t turned it on, and change startproject to insert that setting by default.

6 Likes
2

Yes please! Especially with scoped includes, for me they can get very long and very hard to read:

{% include "client/visitorcount/store_visitorcount_overall_statistics.html" with overall_statistics=overall_statistics overall_statistics_json=overall_statistics_json selected_store=selected_store user_stores_with_sensors=user_stores_with_sensors chart_height=weeks_comparison_chart_height only %}

Would rather have this look like:

{% 
  include 
    "client/visitorcount/store_visitorcount_overall_statistics.html" 
  with 
    chart_height=weeks_comparison_chart_height 
    overall_statistics_json=overall_statistics_json 
    overall_statistics=overall_statistics 
    selected_store=selected_store 
    user_stores_with_sensors=user_stores_with_sensors 
  only 
%}
2 Likes
3

After seeing this example, it’s a clear +1 to me.

1 Like
4

I assume this would affect all usages of {% %}?

If so a possible side effect is that prettier formatting would no longer break Django templates.

+1 from me

5

+1 from me.

This feature has been looked at several times in the past. @thibaudcolas 's django-developers post from 2 years ago has links to the past attempts: https://groups.google.com/g/django-developers/c/rrDlBF4-uY4/m/JlcmjWC_AwAJ

I think there’s still a question of feasibility. It’s not as simple as just changing the one regex :wink: .

There’s also an open ticket from @theorangeone about that particular regex’s possible poor performance: #35675 (Reduce impact of parsing crafted templates with repeat tags) – Django . Opening it to newlines will only make that possibility worse, but maybe that’s a separate concern.

I think this will be the case, yes.

By “prettier”, I assume you mean Prettier, rather than general formatting tools that make things prettier :wink:

6

+1 – thank you for finding that thread and tagging me, I’d still love that, just ran out of steam last time I tried to make it happen. There wasn’t any particular reason, just got busy with other things.

7

:smile: yes I do mean the tool, not just things being prettier :joy:

8

I see there’s an existing PR linked in that mail thread: Multiline templates by loic · Pull Request #2556 · django/django · GitHub which honestly looks pretty good. I think we’d want a bit more paranoia when it comes to backwards compatibility like I mentioned above though.

Or am I the one being overly paranoid now?

9

I wonder if we could use django-upgrade to solve that backwards compatibility issue?

10

How do you mean solve it?

11

django-upgrade isn’t in the templates game, but Djade is :wink:

We could do this to “escape” any existing multiline non-tags:

-{% This isn't meant to be a tag
+{{ '{' }}% This isn't meant to be a tag
-nor this %}
+nor this %{{ '}' }}

Happy to add this to Djade, if need be.

But then that doesn’t really fix adoption.

Perhaps a better approach would be a Python-style “future load” to initially opt into the multiline tag feature: {% load multiline from future %}. We can flag the deprecation warning for any templates that contain unmatched {% / {{ / {#. Then a later version makes the behaviour default.

1 Like
12 
{% 
%}

Would be automatically transformed to

{% verbatim %}
{% 
%}
{% endverbatim %}
13

I think you want {% templatetag openblock %} et al.

14

Changing the parsing part way through the template seems fraught!

I would object to warning based on the complaints that the templates are already too slow. If we do the warning, surely they would be even slower.

15

a clear +1 from me.

I’m also for some sort of tooling that would detect that during the upgrade process but if django-upgrade is not that tool, I’m not for asking people to run X different tools to upgrade something

1 Like
16

Is there any such tool for anything in Django today? I can’t think of any…

17

+1 from me! That would be a great improvement eg on scoped includes.

18

I also have been wanting this for many years. At work we use a lot of includes and other tags with multiple often quite long strings passed in (sometimes template names and so).

I think there’s clear rationale and consensus for it and a PR that could hopefully be revived, so I think someone needs to step up and write the ticket and open a new PR.

1 Like
19

It doesn’t need to be part-way through. Like Python’s __future__ rules, we can require that the special tag appears near the top of the file.

The change here is in tokenization, which is technically the step before parsing.

The warning needs only be done once when loading the template. After tokenizing, when the old mode is in effect, we could check all text nodes for {%, {{, or {#, and warn for each occurrence. I would expect this to be a small cost compared to the existing tokenization run.

The “templates are slow” statement/meme normally refers to render time, not load/parsing time, as far as I’ve heard. So, I don’t think such a check would impact the main cost.

(I have an open proposal to help optimize render time a bit: Add __slots__ to template Node classes (only) .)

20

A rough first draft.

1 Like