Rejuvenating vs deprecating Form.Media

Django Internals
1

Apologies, this is going to be long. There is an accepted ticket from 9 years ago, about renaming or removing Form.Media.

I think it’s worth revisiting this issue, and ideally pushing it forward: Media feels like a rather sad and neglected part of Django at the moment. It’s not part of all the recent template-based form improvements, as far as I can tell. It’s fairly complex for what it tries to do, especially with the magic inheritance by way of metaprogramming: if you inherit from a class that has a class Media, and your new inherited class also has a class Media, the two get magically merged (which is the only reason all Widget subclasses have a metaclass, incidentally).

The feature has also not kept up with changes in front-end development. There is no way to define that a provided script should be loaded with async/defer, which is a very common use case. We don’t support nonces or integrity attributes for security. On the docs side, we list deprecated media types like tty, tv aural etc, some of which have been deprecated for a while, which also highlights how odd it is that this part of Django should care about the W3C’s media query updates. (Side note: this is really only a docs issue, as Django does not validate those keys.)

I wish we could get rid of it entirely, but I suspect we really should keep it or something like it around for the admin (which means keeping around all the merge logic in metaclasses etc, which isn’t ideal, but at least a compromise?). The whole thing feels just very out of place in Django – I can see that it’s a janky-but-useful way of extending the admin, but for regular form.

To play devil’s advocate: The feature is not entirely unused – GitHub finds 8.7k results, 4.7k of which appear in some sort of admin. It’s also in django-CSM, though I didn’t look into it in detail to see how recommended or used it is there.

As far as I can see, sensible options are:

  • Deprecate and remove Media entirely and have some replacement in admin to allow people to add scripts and styles
  • Deprecated and remove Media from Form, but keep it in admin sites (not much maintenance advantages here)
  • Add {{ form.media }} to default form templates – and maybe change it so that we support things like defer and async in the js attribute.

I’m posting this here in the hopes of hearing about general use of Form.Media and maybe finding a consensus on how to move forward.

3 Likes
2

The Wagtail maintainers will probably want to chime in here since Wagtail seems to be a heavy user of forms.Media. I don’t use Wagtail myself but I like using forms.Media to add CSS and JavaScript to the frontend. In the past I have used it (a lot) for websites, not just for the admin, but these days it’s mostly related to the admin.

django-js-asset is linked in the Trac ticket as well. Adding additional attributes such as defer, async or even arbitrary attributes is relatively easy with it. It is being used in several third party packages such as django-mptt, django-ckeditor etc. to ship configuration to the JavaScript code without having to override or add any HTML templates; this made it much easier to support multiple versions of Django in the past. Maybe a widget template would be the way to go these days, but I’m not sure if that would be so much better.

I don’t have a better idea to contribute right now. I’m quite happy with the status quo and the js-asset hack, but I’ll certainly keep an eye on this thread to see what a better future might look like.

1 Like
3

Thank you for brining that up – I was aware only of django-cms, but if wagtail use it too, then I think that’s a further good argument against deprecation or limiting it to the admin entirely.

What I’d mostly like to fix is how Form.Media feels neglected and out of step with current Django and current web features in general. I do think including {{ form.media }} in form templates by default and supporting added attibutes for scripts would be enough to bring it back into cohesion (as I think using a third-party package just to get defer and async in is really really not great if we’re committed to having Media stay!). Changed post title to reflect this!

Does anybody else have opinions on default-including {{ form.media }}, and allowing scripts to be an iterable of dicts rather than just strings, in order to provide more rendering arguments?

1 Like
4

Hi all :wave:

I posted a rather long and rambling post on the mailing list a while back about this ticket. This dates back when I first started contributing. More import than my email was that it got a few thoughts from more knowledgeable folk.

https://groups.google.com/g/django-developers/c/Hjci3SyDx9k/m/Dd5qTXSTAwAJ

Although that was a fair amount of time ago now and I’m not sure it clearly addresses the issues brought up here.

2 Likes
5

For what it’s worth we are trying to push ahead with CSP compliance and also easier client-side code customisations (in the admin) for those building with Wagtail.

The form / widget media is a really nice approach for providing scripts and their associated data attributes in the same area of code. An example is a Django field widget that adds Stimulus data attributes and the relevant script needed. This approach is convenient, builds on Wagtail code primatives and does not require any template driven dynamic inline scripts.

However, providing an ergonomic way to go further with nonce attributes and even module scripts would be really appreciated.

This means we can provide documentation for an easy way to add custom JavaScript widgets, integrate with existing ones and also make the code easily CSP compliant plus performant.

See some of the related Wagtail references.
CSP compatibility issues · Issue #1288 · wagtail/wagtail · GitHub - CSP
rfcs/078-adopt-stimulus-js.md at main · wagtail/rfcs · GitHub (RFC for Stimulus usage, and CSP / customisation goals).

I’m on the core team with Wagtail but maybe some other core team members would have some additional context to add.

1 Like
6

Btw, feincms uses it too but that’s a given since django-js-asset was extracted from it.

The implementation of forms.Media has been cleaned up a bit by Claude Paroz in Fixed #29490 -- Added support for object-based Media CSS and JS paths. · django/django@4c76ffc · GitHub , if the JS object has a __html__ method it’s being called. So, a dict based approach would be strictly worse IMO than what we have right now.

7

I came here to link to this :blush:

I’m pretty sure @claudep has thoughts in this area too (though my searching for them this morning has failed me :thinking:)

8

@mathiask mentioned the fix for #29490. You can see in the test part of that commit the idea behind using object-based assets, so you should be able to add async/defer stuff rather easily with this approach. Pushing something like that in the core forms code was my hope at some point, but it’s hard to convince the dev community (it was argued that this might be better left to 3rd-parties).

9

Just adding my two cents as another contributor to Wagtail – I’d agree it feels like an API that’s not kept up with front-end development. Both in terms of standards (async scripts, CSPs, ES modules, web components) but also practices (dependency management, transpiling, module bundling).

The API works well when loading standalone widgets (for example Wagtail’s Handsontable integration), doesn’t work well when a suite of widgets is implemented with shared code.

For CSS in Wagtail, we’ve only kept Media where widgets have very specific styles. Almost all widgets are built on a shared set of UI components (some form widgets, some more broadly reusable), so the forms-only Media paradigm isn’t a good fit.

For JS – we’re using Media more, but I’d guess mostly due to inertia. We’d ideally break up our JS code into more granular files, but it’s a lot of work to refactor this for modern practices. The best results for this with our current tooling would be code splitting and dynamic imports, which we could combine with Media to some extent but the value isn’t super clear to me – we could just as well do this on a much more granular level without using the Media API at all.

Back to what to do with it – my main recommendation would be to rejuvenate it with modern standards and practices in mind, at least documenting how to use it in a way that makes sense with:

  • script type="module" and dynamic module imports (import() in JS)
  • Import maps (perhaps doing some kind of imports map merging?)
  • async and defer attributes
  • nonce and integrity attributes

So again with Wagtail’s Handsontable integration in mind, currently the Media API helps us produce the equivalent of:

<!-- {{ form.media.css }} output -->
<link href="/static/table_block/css/vendor/handsontable.css" rel="stylesheet">
<!-- {{ form.media.js }} output -->
<script src="/static/table_block/js/vendor/handsontable.js"></script>
<script src="/static/table_block/js/table.js"></script>


[…]
<!-- {{ form.my_table_field_1 }} output -->
<input type="hidden" id="id_my_table_field_1" value="[…]">
[…]
<!-- {{ form.my_table_field_36 }} output -->
<input type="hidden" id="id_my_table_field_36" value="[…]">

Instead we’d want to use ES modules with the relevant attributes and import maps, so our table.js script can do its own loading of Handsontable (and any other dependenceis) only as needed with import():

<!-- {{ form.media.css }} output -->
<link href="/static/table_block/css/vendor/handsontable.css" rel="stylesheet">
<!-- {{ form.media.importmaps }} output -->
<script type="importmap">
{
  "imports": {
    "handsontable": "/static/vendor/handsontable.js"
  }
}
</script>
<!-- {{ form.media.js }} output -->
<script async type="module" nonce="nonce-[…]" src="/static/table.js"></script>

[…]
<!-- {{ form.my_table_field_1 }} output -->
<input type="hidden" id="id_my_table_field_1" value="[…]">
[…]
<!-- {{ form.my_table_field_36 }} output -->
<input type="hidden" id="id_my_table_field_36" value="[…]">

And as a bonus to go further, add Web Components to the mix to guarantee styles are encapsulated:

<!-- {{ form.media.templates }} output -->
<template id="handsontable_widget">
  <link href="/static/vendor/handsontable.css" rel="stylesheet">
  <input type="hidden" value="[…]">
</template>
<!-- {{ form.media.importmaps }} output -->
<script type="importmap">
{
  "imports": {
    "handsontable": "/static/vendor/handsontable.js"
  }
}
</script>
<!-- {{ form.media.js }} output -->
<script async type="module" nonce="nonce-R4nd0m" src="/static/table.js"></script>

[…]
<!-- {{ form.my_table_field_1 }} output -->
<handsontable-input id="id_my_table_field_1" value="[…]"></handsontable-input>
[…]
<!-- {{ form.my_table_field_36 }} output -->
<handsontable-input id="id_my_table_field_36" value="[…]"></handsontable-input>
2 Likes
10

Just a quick update on this to say Wagtail now has a clearer path towards adopting the modern standards I mentioned above, in particular import maps and declarative shadow DOM for Web Components. Import maps are now supported in the latest versions of all the browsers we want to support, and our browser support targets will allow us to use the technology in November 2024. Declarative Shadow DOM isn’t supported by Firefox currently but I have hopes it’ll also be available there by November 2024.

We’ve not had planning discussions about this yet, but I think it’s crystal clear those technologies would help us with extensibility of the admin interface and improvements for compatibility with CSPs (which @lbee touched on above). I suspect they’d also help us make Wagtail faster and more energy-efficient (being more clever about what we load in the browser when).

I’d assume they’d be helpful to many more projects in the Django ecosystem, so keen to make it happen not just for Wagtail if people are interested.

With that said, I’m not clear on what’s possible. Even with being aware of those requirements a year ahead of time, it’s still not enough time for us to make the changes and have them be available in the supported versions of Django which we have to support.

As I understand it, if we made changes happen in django/django in time for the v5.1 release code freeze ahead of the final release in August, we’d only be able to rely on them once v4.2 is no longer supported in April 2026 (so almost two years behind schedule for us).

So I suppose the best option would be to create our own API that bridges over compatibility until then? What do people think? Would you want to collaborate on changing this in django/django? Does this feel like it will require a DEP?

1 Like
11

Hey @thibaudcolas — thanks for the follow-up!

Yes… so Django’s release cycle (being so slow) means that it’s always going to be the case that we (i.e. all of us) need to do something (a backport essentially) for the current versions (the current LTS really) and then migrate to whatever we can get into Django as that becomes available. (In frontend that’s like a polyfill yes?)

I’m not quite sure the API you have in mind… :thinking:

I was reading it as (maybe) that adding form.media.importmaps would be sufficient but then you write…

… so something else, presumably more powerful?

(What follows assumes the options to Just update Media somehow don’t really fly… — Q: Is there a quick change available there?)

My current work is leveraging the (incoming) per-field (group) template that are part of Django 5.0. That’s proving a lovely pattern, allowing customising the form inputs one-by-one. The thing that’s missing there (and Media related) is specifying the dependencies that each widget needs loading.

So, modernising that, and handing different dependencies, and minimising what’s loaded, and all that jazz sounds awesome. (import_maps in particular seem a real boon.)

This feeds directly into what I’m working on, so I’d love to be involved.

I can’t believe that this is something we’d be able to get right first time, or that experimenting wouldn’t benefit from the fast release cycle of (a) third-party package(s).

I’d lean towards trying to develop prototypes/experiments/ideas outside of Django itself until we have a clear idea of what’s needed. (Unless you already have it? — which you might :sweat_smile:)

A draft DEP might well though be a good place to coordinate efforts: it would be good to have somewhere to gather A) Goals, and B) Work so far at the very least — making sure we can coordinate is going to help make progress.

So that’s what comes to mind. Exciting!

If I’m totally missing the boat on anything, please do say: it’s not always easy to know where you’re at. :heart:

2 Likes
12

Thanks Carlton :relieved: I don’t have any API in mind just yet. I can certainly do a bad first draft. And related API design activities, with help from others here if they’re keen :grimacing:

  1. List requirements further than in this thread
  2. Try to figure out who to get involved
  3. Perhaps research the ecosystem further like @rixx started
  4. Research prior discussions further (link from @smithdc1)

As far as next steps – it looks like we might have enough info here to draft a DEP with sections missing. Then try out implementations in a package. Then update the DEP based on those implementations. Does that sound right?

And perhaps as part of this process, early on, we can identify possible quick wins?

@matthiask it looks like you’ve gone further on this than most with django-js-asset – could we get your thoughts on this again, based on discussions since your last message?

13

Yep, this :sweat_smile:

Sounds like a plan. (I’m all for bad first drafts :stuck_out_tongue_winking_eye:)

1 Like
14

Yes sure, thanks for asking! I mostly use django-js-asset for loading assets in the Django admin. I have also used forms.Media for the frontend in the past, but I’ve stopped doing that many years ago. I’m a happy user of vanilla JS, webpack (yes) or esbuild these days when it comes to building stuff for the web.

I have mainly used django-js-asset to ship additional data to the admin scripts so that I can avoid overriding django.contrib.admin templates because that meant I have to reenact all changes to Django proper in my own packages and that became very annoying once I started maintaining too many packages. So, I consider my own itch scratched for the time being and I’m not personally too interested in modernising the frontend code for my admin widgets all that much. There are discussions going around on whether the Django admin can and should be considered a CMS (my opinion is that it is and should be) but as long as there isn’t a broad consensus that the Django admin should evolve I’m not sure if a rework is in order or not.

I don’t want to discourage anyone and I really do not want to sound like an old timer here but apart from the arguably confusing name (media vs. static) I’m quite happy with the status quo in this area. I think that the current implementation hits a sweet spot, sort of like a local maximum, where you would have to put in a lot of energy to get to the next local maximum, which additionally is already occupied by much more powerful^Wcomplex frontend building tools.

2 Likes
15

That was fast! :smiling_face: thank you. I quite like the metaphor of the local maximum. Can certainly be the right place to be sometimes.

It feels like a common use case to add additional attributes to script tags like django-js-asset does, would you not want to see that added to Django? You called it a hack earlier, but it seems like it’s a widely-used one and possibly API-compatible hack, so why not take it further / do it in Django itself?

Or if not by changing the implementation in Django, perhaps mention in the Django docs “here is a way to add attributes which would help for use cases X, Y, Z”?

There are discussions going around on whether the Django admin can and should be considered a CMS (my opinion is that it is and should be) but as long as there isn’t a broad consensus that the Django admin should evolve I’m not sure if a rework is in order or not.

Could you share some of those discussions? Personally I think there’s room for improvement on Form.Media regardless of whether those improvements would be beneficial for the Django admin or not, but keen to have more context here to understand what others requirements people might have in mind.

16

Yes, I think that would definitely be useful. I’m sorry for being terse earlier. It definitely uses an ugly implementation before Django 4.1, but since Django 4.1 introduced the mentioned object-based assets it isn’t all that hacky anymore, and I would certainly prefer it if the JS object was a part of Django. I would also like it if it supported inline scripts even though that makes CSP compliance a bit harder I think. (I wrote down my thoughts here Django admin apps and Content Security Policy compliance - Matthias Kestenholz )

Sure, here’s a recent conversation on Mastodon about this: jacobian: "@wagtail@fosstodon.org @FunkyBob@chaos.social @pa…" - social.jacobian.org , the top post links to Why is the Django Admin “Ugly”? — CodeRed which outlines a few reasons why the Django admin is (relatively) ugly and why everyone should go and use wagtail (nothing against wagtail / no shade intended / no hard feelings / etc.).

I have also read somewhere the idea that the Django admin is basically a slightly better PHPMyAdmin. I don’t remember where.

I still remember the “trusted users editing structured content” from the django book during the 1.0 days (or something) and that has definitely formed my thinking. I would very much like better undo functionality, and some support for some task-centric workflows. But those aren’t essential features for the way most of our clients use the CMS^WDjango admin. I’m just sad that the admin looks a bit dated and that makes for a bad first impression which lets some people oversee that it’s actually a really powerful and quite usable CMS in its own right.

17

Thanks! I thought the use of the __html__() method was what you considered hacky, that makes much more sense now :slight_smile:

And yeah I actually saw that thread, and did that cheeky reply with the Wagtail account, and then promptly forgot about the thread. Something to mull over a bit more.

18

No worries! It’s not a new discussion, just the newest instance of it which came to mind :slight_smile:

19

Hi all, here is my bad first draft: DEP 84: Rejuvenate form media. I’ve kept it as simple as I could:

  1. Adding a list of all the possible things we might want to support
  2. Adding good inspiration from the open source ecosystem for us to learn from

There’s no API design yet, I’d like to spend more time reviewing possible requirements first.

If you want to help with this, here are five specific tasks which I think would be really helpful right now:

  • Add more possible requirements
  • Review awesome-django for packages with form media-related functionality.
  • Review Django Packages for packages with form media-related functionality.
  • Update awesome-django-perfromance with existing performance-related Django packages relating to form assets.
  • Also update awesome-django with good packages (also security-focused ones)
1 Like
20

Hey @thibaudcolas — thanks for that. I had a look; let me ponder it.

I think one thing we could clearly state are non-goals, saying what we’re not going to do. Something too complex (To Be Defined) or too closely tied to a specific option from JavaScript land (a specific bundler pipeline, say) should be punted to Third Party Packages. (This is why compressor or pipeline were never candidates for core.)

I like your focus on updating the resources that highlight the available options.

One possible outcome could be ≈ The ecosystem handles this well; options A, B, C do well, but aren’t suitable for inclusion in Django; we’re going to deprecate Form.Media. (Note, “could be”.)

Will ponder.

Merry Christmas :christmas_tree:

1 Like