Gradually adopt a modern JavaScript framework for use with Django admin

Django Internals
1

jQuery is great, but it lacks the reactivity that makes modern JavaScript frameworks like Vue.js or React so powerful for building dynamic, reactive experiences that people expect from the modern web. This calls for eventually replacing jQuery with a modern reactive framework in Django admin.

I’m not sure if this has been discussed before. If it is, now might be a good time to revisit this topic as the Vue.js maintainer recently released petite-vue, an official alternative distribution of Vue.js “specifically optimized for “sprinkling” small amount of interactions on an existing HTML page rendered by a server framework” - exactly what’s needed in Django admin.

Vue.js is used by large venerable projects like Wikipedia’s MediaWiki and GitLab that use server-side rendering, for its simplicity, ease of use, as it is very easy to integrate into this setup, and as it can replace jQuery gradually without requiring a large up-front rewrite. Now that petite-vue has emerged specifically for the server-side rendering use case, there’s even more reason to consider it.

Alpine.js is addressing a similar scope, but petite-vue is around half the size of Alpine and there are more people familiar with Vue.js. At 5 KB, petite-vue is 17x smaller than minified jQuery 3.5.1 (88 KB).

Vue.js can be easily used side-by-side with jQuery without affecting page performance or causing side effects, so I suggest a gradual componen-by-component replacement as needed. This is how GitLab did it and they seem to be very satisfied with their choice, their UI tech lead Jacob Schatz writes: “Vue works really, really well, when used properly. It turns out Vue isn’t a buzzword, Vue is a workhorse. A lot of our problems have been solved, by us and others.” Also, there’s a thoughtful and thorough consideration why Wikimedia Foundation chose Vue.js here.

What do you think, would this help Django forward?

2

Personally, no.

The Django admin is not the “front-end” for your application. Directly from the docs

The admin’s recommended use is limited to an organization’s internal management tool. It’s not intended for building your entire front end around.

I would not want to see the limited time / effort / energy available to those most active with the framework need to be diverted to such an effort - in the absence of an existing third party library demonstrating the benefits.

In other words, I’d like to see someone implement a custom admin facility that demonstrates such a migration is clearly and obviously “better” than the current implementation before I would want to see Django itself adopt such a migration.

1 Like
3

Fair point. Would a GSOC 2022 project fit the requirement to demonstrate such a migration is feasible and useful without diverting the limited energy available to maintainters?

4

I wanted to acknowledge your response / question, but a real answer needs to come from someone more knowledgeable than I. As an opinion, I think it’s better suited as a separate initiative.

There are a variety of packages available that provide alternatives to the standard admin site, see https://djangopackages.org/grids/g/admin-interface/. It might be worth evaluating one of them as a basis for further enhancements as opposed to starting from the default package.

5

As a technical board member I would be against trying to move the admin to any JS framework. Django has outlasted (the peak popularity of) many “cutting edge” JS frameworks, so we cannot bet on any of them continuing to exist and be maintained.

And indeed as Ken noted, the admin is not for developing your “core application” in. You’re free to use Vue, htmx, Alpine or whatever you feel like for your own admin extensions. But we wouldn’t want them in core, dependend on by Django’s wide userbase.

2 Likes
6

Speaking without my hat as technical board member (because I haven’t put to much thought into this yet) I would not immediately be opposed to a JS framework. The main question to answer first though would be: What would we gain and what would we loose? We should not forget that the admin is not a SPA with only limited usage of JS, it mostly does fine without JS whatsoever.

Assuming that we would decide on using a framework there are more things to consider. I do not buy the size argument. The js file can be cached easily and time to first paint (or whatever) is not really relevant for the admin. With size (mostly) out of the way it would then make sense to focus on something with minimal dependencies and not requiring a complex buildchain.

Last but not least, from a purely security point of view it has to fully work with a strict CSP (I guess most do, but just throwing it out).

7

I’m sorry, the following answer is long. In short, maybe now is indeed not a good time for adopting a new framework (or library), but we should do long-term strategic planning how the admin site should evolve to support dynamic page updates to keep up with the modern web.

The main question to answer first though would be: What would we gain and what would we lose?

The losses have been already explored somewhat, let me bring up some gains by quoting the excellent reasoning of the Wikimedia Foundation folks from their Adopt a modern JavaScript framework RFC as they are in a somewhat similar (albeit seemingly more chaotic) situation technology-wise, using jQuery and server-side rendering:

The need to evolve our platform is very evident when it comes to how we design, develop, and deliver experiences to users in the browser.

WMF projects still rely on the same approach to front-end development that they did a decade ago: most user interface elements are built in PHP (often without relying on templating languages), and enhanced in the browser using jQuery and our own in-house frameworks (OOJS and MobileFrontend) and UI component library (OOUI). There are many pain points associated with this way of working in 2019.

Thoughtfully adopting a modern JS framework will provide many immediate improvements to the front-end development process at the Foundation, and will help to pave the way for further architectural improvements in the future. Some advantages of using a modern JS framework include:

  • The ability to build UIs in a declarative way (describing what to build), as opposed to the present imperative approach (describing how to build it), leading to more concise code that is easier to reason about
  • Better support for breaking up UI code into small, manageable components with clear separation of concerns
  • Components built with a modern framework would be reactive (automatically responding to user input) out of the box, with no need for manual DOM reconciliation. This can potentially eliminate an entire class of bugs
  • State management systems (Redux/Vuex/etc) would be available to help make complex data flows consistent and predictable
  • Unit-testing of UI elements would be greatly simplified, leading to better test coverage and fewer bugs in our UIs
  • The ability to do server-side or isomorphic rendering of UI elements opens up many new architectural possibilities that could be explored in the future
  • Aligning front-end development within the Foundation to current best-practices in the larger web will simplify onboarding of new developers (a process that is currently quite painful and time-consuming, since many things need to be re-learned here) and lower the barriers to contribution generally
  • Relying on tools that have large, active communities outside the Foundation will let us benefit from the larger community’s work (fixing bugs, writing documentation, maintaining the framework); this will also make it easier for innovations developed within the Foundation to be adopted across the wider web.

I like Adam’s thought that Django has outlasted many JS frameworks and should therefore steer away from them. However, I would like to point out that we already use jQuery and bring in more and more complex external libraries like Select2 for autocomplete, driven by pressure to "deliver [better] experiences to users in the browser".

I use and have seen others use a common Django project architectural pattern where

  • the core end-user application is a SPA using a modern JavaScript framework (Angular, React or Vue) and ready-made UI components (e.g. Vuetify with Material Design components) that uses either the Django REST Framework or Graphene to access data,
  • the back office management application for staff is generated with Django admin and is usually customized both in the server- and browser-side,
  • there is a reports module that is raw SQL-heavy that may exist as separate, fully custom views in admin (see example here).

This has been a very successful application development model and although the admin is not the core front-end of the application in this approach, its full power and customizability is crucial for effective back office application development. Usually the data structures are complex and data volume is considerable, so, for example, autocomplete has to be used in foreign key fields to avoid loading massive amounts of data during page rendering etc.

In practical terms, here are a couple of use cases that I would argue should be easy to achieve in the admin site, but are currently not:

  • You build a book publishing site for a publisher. Books and authors are managed in Django admin. One book can have many authors and the staff would like to see and edit author information in the book change view. As there are tens of thousands of authors, you have to use the Django admin autocomplete multi-select field. How can you show the standard related object popup, like when clicking the ordinary ForeignKey field widget pencil icon 🖉?
  • Authors live in different countries and you have to enter the municipality/city where they live. How can you filter the municipalities autocomplete by the country drop-down as the selected country changes?
  • When you issue the invoice for a book, you select the customer from the customers autocomplete. How can you update a HTML element on the page to show if the selected customer requests an e-invoice?

These are just examples, I hope you get the general point - as data volumes and complexity grows, solutions like autocomplete, dynamically updating parts of the page as selections change and elaborate components become more and more relevant. This evolutionary pressure has caused the emergence of modern JavaScript frameworks, design systems and UI component libraries and it will affect the relevance of Django admin - should one not use Django admin and rebuild the admin site as a SPA instead when needing this?

It would be interesting to hear what architectures do you use in large production applications and how do you manage the complex and dynamic requirements for the admin site.

I do not buy the size argument. The JS file can be cached easily and time to first paint (or whatever) is not really relevant for the admin.

Agreed. I brought up the size argument to emphasize that it does not affect page rendering speed like full-blown SPA frameworks would and can harmlessly exist side-by-side with jQuery.

With size (mostly) out of the way it would then make sense to focus on something with minimal dependencies and not requiring a complex buildchain.

Neither petite-vue nor Alpine.js have dependencies nor require a build step. Just including them is sufficient. Angular and React are out on that regard, as they both require a build step (React can also be used without a build tool, but it falls back on its imperative CreateElement API which is not ideal for general use). Vue.js can, however, be used without a build step as well, that’s one of the reasons why Wikimedia Foundation chose it.

Last but not least, from a purely security point of view it has to fully work with a strict CSP (I guess most do, but just throwing it out).

Good point, thanks for bringing this up! Quoting petite-vue docs:

petite-vue evaluates the expressions using new Function() , which may be prohibited in strict CSP settings. There is no plan to provide a CSP build because it involves shipping an expression parser which defeats the purpose of being lightweight. If you have strict CSP requirements, you should probably use standard Vue and pre-compile the templates.

Likewise with Alpine.js:

In order for Alpine to be able to execute plain strings from HTML attributes as JavaScript expressions, for example x-on:click=“console.log()”, it needs to rely on utilities that violate the “unsafe-eval” content security policy.
In order to accommodate environments where this CSP is necessary, Alpine offers an alternate build that doesn’t violate “unsafe-eval”, but has a more restrictive syntax.

8

As just one perspective on this.

We reserve the Django Admin for only performing low-level operations where no other facility is necessary. All other “back-office” processing as you describe it is done with standards views / pages.

We fully and completely embrace the philosophy as described in the admin docs:

If you need to provide a more process-centric interface that abstracts away the implementation details of database tables and fields, then it’s probably time to write your own views.

Our back office doesn’t need and would not materially benefit from any kind of SPA-style application. These aren’t people needing to “be sold” on the site or the functionality provided by it. It needs to be fast and targeted on what they need to accomplish as a part of their job. They’re not choosing to use what we provide, it’s required. Our job isn’t to make it look “slick” or provide “the best UI/UX” - it’s to allow them to do what they need to do with a minimum of effort.

(Yes, I’m rather curmudgeonly about such things, but I find many web sites I visit to be effectively unusable. Too much “fluff and style” and not enough function.)

9

Thanks, @KenWhitesell for sharing your perspective! I agree that Django admin should keep a minimalist aesthetic and avoid excessive fluff. However, I would argue that not all of modern web is fluff, there are real needs that drive innovation, for example the use cases in my previous post.

I kept on thinking in the bounds of Adam’s and Florian’s requirements:

  • Django should avoid JavaScript frameworks, so Angular, React, Vue and others are out,
  • strict Content Security Policy (CSP) support is a must, so Alpine.js and petite-vue are out,
  • the solution should have minimal dependencies and not require a build step,
  • in general, the solution should stick as close to the web platform as possible, should be stable and have long term support,
  • it must be possible to adopt it gradually, side-by-side with existing jQuery code.

On the other hand, the solution must be powerful enough to support the use cases in my previous post in a straightforward, succinct way, and have the advantages outlined in the Wikimedia Foundation RFC.

Maybe Web Components fit the bill? It would probably make sense to additionally use a thin library like Lit by Google to simplify building reactive web components. Lit can be loaded as a JavaScript module without a build step and adheres to a strict CSP. Here’s a working example:

<!doctype html>
<html>
<body>
  <script type="module">
    import {html, css, LitElement} from 'https://unpkg.com/lit?module';

    export class SimpleGreeting extends LitElement {
      static get styles() {
        return css`p { color: blue }`;
      }

      static get properties() {
        return {
          name: {type: String}
        }
      }

      constructor() {
        super();
        this.name = 'Somebody';
      }

      render() {
        return html`<p>Hello, ${this.name}!</p>`;
      }
    }

    customElements.define('simple-greeting', SimpleGreeting);
  </script>

  <simple-greeting name="World"></simple-greeting>
</body>
</html>

Besides Google, Lit is used by Adobe, Cisco, IBM, Red Hat, SAP, VmWare and others.

10

my 2 cents… or how about simple vanilla? http://vanilla-js.com/ (humour inside)

As a heavy and frequent user of the django admin, I would really be philosophically bothered by it being made into some sort of SPA/hybrid. It could definitely improve by shedding (more and more useless?) weight like jquery and gulpifying/other all the css/js. But SPA feels wrong to me. Heavy use of gracefully enhanced xhrs/fetches or even websockets for navigation would be light, faster and bring enough of that “modern fluff” to the /admin/ UI, imho.

11

I think it would be acceptable to use some web components, but I am not sure if there’s a specific place to adopt them in the admin right now. And naturally Django doesn’t need to use webcomponents for you to add some to your customized admin.

The autocomplete dropdown, Select2, is probably the most complicated bit of JS in the admin. If there was an alternative built as a webcomponent, we could look at switching to it I guess. It would be nice to know the feasibility of that.

12

Glad to hear! I’ll keep an eye on Select2 web component alternatives and will reply once I have found a feasible solution.

Sure, I agree that the time is ripe for replacing jQuery with vanilla JavaScript. HubSpot maintains a nice guide that translates jQuery idioms to vanilla JS and here’s a blog post by GitHub engineers that documents their jQuery to vanilla JS (and web components) rewrite. However, building advanced components like Select2 with vanilla JS is not feasible.

One more thing: I think it is important to keep in mind that web components don’t belong to a framework, they are part of the web platform. Libraries like Lit exist only to make component authoring experience pleasant and efficient. Here’s a great write-up on this topic by Cory LaViska.

13

About readability, for me, inlines.js is the most complicated.

I already needed to use nested formsets, I was successful doing with vanilla js. Web components wouldn’t be a bad idea either.

14

I started my move away from jQuery in templates needing UI customization in early 2019, and I’ve been thinking about the role of modern JavaScript in Django ever since.

I am in full support of a vanilla JavaScript implementation of admin, but also recommend careful consideration of the messaging and documentation that would accompany such a release.

Moving away from jQuery is a teachable moment for developers who use Django and have avoided adopting modern front end. Many developers may not realize JavaScript now effectively does everything jQuery did, I was one of them. They may not know that even the trusty Bootstrap library has ditched jQuery entirely.

Since learning ES6 and its patterns, I’ve done a customized adaptation of Django to support development and deployment a React app inside a view-rendered template. This was a “one view” SPA in a Django App.

That experience, along with helping maintain django-address (which has a jQuery dependency) made me think more deeply about Django’s influence and requirements.

I realize excluding FE frameworks as a basis for replacing jQuery in admin has already been given up in this thread. As I said, I think that is the right choice.

However, I think there is a strong case to be made for Django keeping a big tent and having open arms for API-driven front ends.

Whether it be React or something else, Django should have great documentation for how, and strongly consider incorporation into contrib or core something like django-js-reverse, which makes URL routes from Django easily available inside a modern toolchain.

Django documentation should offer specifics on how to encorporate modern JavaScript toolchains and workflows in web development.

To be silent, or continue to refer to past static asset documentation will be a demerit against Django in selecting between it, Flask, Fast API + SQLModel or some other future Python-based framework.

I do not think most Django apps of today need SPAs or want SPAs in their Django app. Existing projects most likely sprinkle JavaScript in on certain pages, maybe use <script> to also include a dependency like moment.js from a CDN URI.

However, doing this also does not take advantage of things like tree-shaking, bundling and minification.

Django should have some documentation recommending patterns that handle this serious gap in modern front end development.

To this end, I recommend not only re-writing Django Admin to use vanilla JavaScript, but to incorporate documentation on how to add a toolchain that at least bundles, minifies and tests the new admin JavaScript.

This documentation should show a pattern for also using the toolchain to build and test JavaScript used in common legacy Django project directory and file structure patterns.

That way, it gives some idea for how Django can work with npm, yarn jest, and esbuild, (or comparable tools) without requiring any of that to make Django work.

One more thing to add on this, is that there is good reason to consider maintaining Typescript as the source on the project. But, as part of the release build, compiled JavaScript is included.

3 Likes
16

yes.

or not and:

. keep simple HTML pages (I know it’s ho-so 1993, but hey it works!)
. enhance the bits that can be enhanced (select2, autocompletes, xhr/fetch/websockets, hell even contenteditable=“true”)

Making django dependent on “heavy” third party frameworks is opening a can of worms similar to npm.