New take on old ticket "Send templated email"

Django Internals
1

Hi folks!

I came across an old ticket (thx Sarah B.!) which targets the developer experience (DX) of creating an email. Since the ticket has been created more than a decade ago, I’d suggest some creative freedom approaching a solution.

As some might know, I’ve taken on improving the DX for implementing emails with my class-based email approach in my package django-pony-express.

I like this approach because it’s very django-esque and easy to use. For the sake of discussion, I’ll post a quick example.

This is how a “regular” email will look like in your code:

class AccountCreatedMail(BaseEmailService):
    subject = "Account created"
    template_name = "account/email/account_created.html"

If you need to add context variable, you can overwrite the get_context_data() method in this class, similar to class-based views. If you need to customize headers, translations etc, there is also a method to overwrite.

In the background there is a HTML-to-plain-text conversion happening which I do via a package. I know that this won’t be portable to Django for multiple reasons but to get started, we could just offer a plain_text attribute and if this is not set, we just strip all the HTML tags from the HTML template and have a basic version to go with. Or we leave it out and you have to provide two templates.

Under the hood, all objects are EmailMultiAlternatives since in 99% of all cases, you want HTML emails if you can help it.

Instead of storming forward and creating a huge PR, I’d like to gather some crowd wisdom on how to find a MVP (GVP? :upside_down_face:) version of this which we could add to Django core.

So, please, feel free to drop ideas, criticism and feedback!

Best from Cologne
Ronny

1 Like
2

My recent experience with an old ticket here on the messages side of things.

There was an existing patch with a large change. To begin with I dug in an produced a large draft PR and opened a forum post. In doing both I was able to see a way forward with a smaller PR (that I am working on now).

Essentially what I am saying is that the way forward for an MVP might reveal itself through the larger PR.

The other realization I had with the above ticket is that the first changes that need to be made are ones that enable the feature to exist in a third-party package. To put this in context, of emails, How would I send a templated email using the existing Django API (send_mail etc)? This is probably a good starting point

3

So there’s two considerations here:

  1. Is it stable?
  2. Should it go into core at all?

On 1, once it’s merged we can’t iterate quickly anymore, and we can’t make breaking changes, so at the very least (for all suggested features) we want to iterate outside of core before merging. So is it stable? Do we want to iterate in pony-express for longer, or did all the issues drop out?

Then 2. We’re trading “batteries included” against the maintainability and velocity of Django itself. The less we have in core, the faster Django goes, and the easier it is to maintain, but the less batteries we can provide. There’s no one right answer there. Lots of vibes towards more batteries, which I think I share, but here’s a nice essay talking about Python which frames the competing view well:

I think I generally lean towards doing something better for folks here in Django. django.core.mail is not great as it is. We’re not likely to remove it, so we should make it fit for use (IMO).

2 Likes
4

I really like the ideas django-pony-express is exploring. I think class-based email could unleash some powerful capabilities, and the approach has a lot of promise.

I also have questions. A lot of questions. The sort of questions that are best answered through prototyping and real-world proof points. You mention “MVP,” and it usually takes rapid iteration to get from an MVP to something more sustainable. As Carlton points out, deprecation and stability guarantees make that sort of iteration difficult in Django core.

One immediate question is, which problem(s) is this trying to solve? Building email from templates? The DX around html+plaintext (with or without a template)? Is there a class/mixin hierarchy like with class-based views (View, TemplateResponseMixin, TemplateView, etc.)? (Oops, that was four questions. Don’t get me started :grinning:.)

Also, it seems like some discussion (here and elsewhere) envisions class-based email as a replacement for Django’s current email APIs. (Maybe I’m misinterpreting those comments.) I’d expect class-based email to build on top of the existing APIs—and that we’d continue improving those APIs along the way. (Just as class-based views didn’t replace function views or HttpResponse. More on that thought in my response over on django-developers.)

You’ve got a great start with django-pony-express. Can you continue using that project to test and refine this proposal? Andy asked a great question: what (if any) changes do you need in Django to push these ideas forward in django-pony-express?

5

Firstly, I like pony-express, great work :clap: :heart: And great presentation last year in Copenhagen :heart:

I can think of some issues/assumptions that would have to be decided before a “core” functionality or battery can be provided:

  • The proposed pattern of django-pony-express is as CBVs are for FBVs. Maybe somehow it’s possible to propose support for both… I don’t know… I’m not gonna argue that I want the FBV-inspired way send_email(template=..., subject=...)
  • Some projects just want plain text: EmailMultiAlternatives is not a one-size-fits-all.
  • Other projects would like to maintain separate templates: The automatic conversion seems volatile.

If you need to customize headers, translations etc, there is also a method to overwrite.

I like these functions a lot, but I think there must be some line drawn between what should be in core and what should not?

If Django itself encourages template-based alternative to EmailMultiAlternatives (which I think should allow for both plain text templates, HTML templates or both), I would think that’s doing the developer a favor. It’s not going to get in the way. But at some point, maybe the functionality should be left for the application or project to define.

Btw. I like the pattern of pony-express, I’ve been reinventing this on a project-basis for a decade :innocent: Maybe I didn’t think I’d look for an application because emailing was always a bit tedious and the problem of doing an OO design seemed smaller than other mail-related problems (consent, translations, cron jobs and retries). Some of these problems are solved with pony-express… which IMO is a very strong case for an app, but not necessarily as a core functionality.

6

First of all, thx to all the ideas and answers! It helped me shape an idea in my head.

I think I’d go for a smaller approach than in my pony-express package because, well, Django is a multipurpose framework and I don’t want to put too much of my opinion in creating emails.

My current idea is as follows:

  • We create a new class called “EmailRenderer” which ultimately creates an EmailMultiAlternatives object
  • This renderer is the place to compose your email object programatically, similar to a view that composes the HTTPResponse object
  • This renderer takes a template name (maybe a plain text template, too) and a to-address.
  • Everything that’s possible (from-header, internationalisation language etc) will be taken from the global settings, as it’s the Django way in every other case
  • If you want, you can overwrite this renderer class to taylor it to your needs (like customise the internationalisation)
  • It’ll be easy to provide a function-based version of this, too, for all the FBV fanboys and -girls

This means that I’d drop the following things from the pony-express:

  • HTML to text conversion
  • Create AND send the email
  • Minor quality-of-life things like subject prefix and default reply-to header

I’d be happy about feedback if I’m heading in the right direction.

Best from Cologne
Ronny

7

Thank you all for this discussion :star:

Tried to have a think about this, looking at @medmunds question of

One immediate question is, which problem(s) is this trying to solve? Building email from templates? […]

I think it’s good to look at how someone might do this in Django already without any extra help.

from django.core.mail import EmailMultiAlternatives
from django.template import Context, Engine

def send_example_email_using_templates():
    template_engine = Engine.get_default()

    email_context = Context({
        "recipient": "Jane",
        "confirm_email_link": "https://confirm_link.com",
        "report_link": "https://report_link.com",
    })

    txt_email_template = template_engine.get_template(template_name="email.txt")
    txt_content = txt_email_template.render(email_context)

    email = EmailMultiAlternatives(
        subject="Confirm email",
        body=txt_content,
        from_email="from@test.com",
        to=["example@gmail.com"],
    )

    html_email_template = template_engine.get_template(template_name="email.html")
    html_content = html_email_template.render(email_context)
    email.attach_alternative(html_content, "text/html")

    email.send()

Looking at this, I assume the pain point is that this gets a bit repetitive when you have lots of emails.

To reduce the boiler plate, we can have something like… (please ignore the naming, this is just to demonstrate)

from django.core.mail import EmailMultiAlternatives
from django.template import Context, Engine

def create_email_from_templates(context, default_template, extra_templates, **kwargs):
    template_engine = Engine.get_default()
    default_template = template_engine.get_template(template_name=default_template)
    default_content = default_template.render(context)
    email = EmailMultiAlternatives(body=default_content, **kwargs)
    for template_name, mime_type in extra_templates:
        extra_email_template = template_engine.get_template(template_name=template_name)
        extra_content = extra_email_template.render(context)
        email.attach_alternative(extra_content, mime_type)
    return email

#----- Updated function --------

def send_example_email_using_templates():
    context = Context({
        "recipient": "Jane",
        "confirm_email_link": "https://confirm_link.com",
        "report_link": "https://report_link.com",
    })
    email = create_email_from_templates(
        context=context,
        default_template="email.txt",
        extra_templates=[("email.html", "text/html")],
        subject="Confirm email",
        from_email="from@test.com",
        to=["example@gmail.com"],
    )
    email.send()

If I understood your proposal about adding a EmailRenderer class, I think it’s trying to solve the same problem (with a different, more class based approach), but please correct me if I’ve misunderstood :pray:

If this is the problem (“too much boiler plate”), is adding to Django core the solution here? Have we ruled out the alternative options?

Alternatives include:

  • add a “Using Django templates for emails” as a “How to” in the Django docs
  • add this to a third party package and await feedback

Also, @theorangeone as you have worked a bit with django.core.mail recently, feel free to also add your thoughts :+1:

8

Is this something which can perhaps be solved in stages instead?

Something akin to a shortcut of send_templated_mail (akin to Sarah’s create_email_from_templates above) which handles a lot of the boilerplate, without needing to decide too much on the specifics of the API in something class-based like pony-express. That should also appease Carlton’s concerns around slow-moving, since even with something class-based, a helper function can still be useful (it’s not like render_to_response is less useful with TemplateView existing).

Sure, it’s yet another API to maintain, but it means we can move fast and slow, at the same time.

9

I am +1 on adding “Using Django templates for emails” to Django’s docs, +1 on eventually adding some sort of class-based email construction API (after prototyping and discussion), and -1 on adding any new template email helper APIs at this point.

Here’s @sarahboyce’s example rewritten with the render_to_string() template helper and new text_body and html_body options I’ve proposed in another thread:

from django.core.mail import EmailMessage
from django.template.loader import render_to_string

def send_example_email_using_templates():
    context = {
        "recipient": "Jane",
        "confirm_email_link": "https://confirm_link.com",
        "report_link": "https://report_link.com",
    }

    email = EmailMessage(
        subject=render_to_string("email_subject.txt", context).trim(),
        text_body=render_to_string("email.txt", context),
        html_body=render_to_string("email.html", context),
        from_email="from@test.com",
        to=["example@gmail.com"],
    )
    email.send()

I feel like this is not a lot of code, and that some version of it would be a useful example in the docs. (Even with added EmailMultiAlternatives boilerplate, if text/html_body gets rejected.) I also feel it’s not really enough code or complexity to warrant a new helper function in Django core.

Did you notice I snuck an extra feature into Sarah’s code? It’s also rendering the email subject from a template. I borrowed that idea from (I think) one of the old Pinax projects, but you’ll also find it in django-allauth’s email templates, django-postoffice, and many other places. I’ve found subject templates simplify my code; others might argue they’re unnecessary complexity.

And that gets at why I’m -1 on adding a new templated email helper. I don’t think we’re going to identify a one-size-fits-all—or even one-size-fits-most—approach. Here are some other examples that come up pretty regularly in practice:

  • Do you want separate text and html templates? Or just html and generate the plaintext from that, per @GitRon’s original suggestion top of thread? Or just text and generate the html, like Django eMark? Perhaps you’d prefer a single template with separate blocks for the subject, text, and html, like django-templated-email? (Or maybe you just want an html-only or text-only message?)
  • If your html uses CSS styles, you’ll probably want to post-process it with Premailer, after rendering before sending. (Because it’s no fun to maintain templates with the CSS already inlined. It’s already messy enough with email still needing table-based layout!)
  • For text body (and subject), you probably want to post-process the rendered message to squash excess newlines and other artifacts of template rendering— &‍amp; you might want to disable autoescape.&‍#28517;

Could we pick some subset for an opinionated send_template_email() helper? Maybe. But if it doesn’t support the kinds of things people need to do in real projects, we’ve just added more complexity and maintenance burden to django.core.mail while still not really making it “fit for use.” And we end up with the same sort of awkward split we have between send_mail() and EmailMessage/​EmailMultiAlternatives, where the simple helper is often insufficient, but the full-featured alternative requires a lot of knowledge to use.

That’s why I think the class-based email idea has promise: a set of well designed classes could support several common cases, while also providing extension points that allow (and make it obvious where to implement) things like post-processing and converting between content types. There’s a lot of prior art in this area: in addition to django-ponyexpress, take a look at django-yubin, django-mailviews, and this TemplateEmail Forge Guide.

Until then, though, I think some how-to documentation on using templates to send email would be really helpful.

[Incidentally—and probably not surprisingly—many of the questions and positions from this thread came up in the original ticket #17193.]

2 Likes
10

I’ve added a basic example to the top of the docs. What do you think about it? Right direction?

1 Like
11

Nice. I like that you’ve surfaced EmailMultiAlternatives at the top of the page, along with the “only for backwards compatibility” send_mail() example that was already there.

At some point, the email docs could probably benefit from some significant reworking. (E.g., moving the APIs people are most likely to need closer to the top; splitting up the “sending and testing email” docs from the “developing email backends” docs; adding some info on configuring email.) But that’s a larger discussion. This PR seems like a nice, incremental change that starts down the right path.

2 Likes
12

Alright, everyone! The PR got merged, thx for all the support I’ve gotten so far!

The question is: Is this sufficient to fix the “old ticket” mentioned above? Or is some functional helper still a valid idea?

What does the crowd think?

Best
Ronny