Custom Email Backend and PicklingError in Django 5.2

Using Django Mystery Errors
1

Hey folks, we are currently on django 5.1.4 and running some local tests to see if we’re good to go to jump ship to 5.2.

We have a minimal custom Email backend that is utilizing Huey for async sending of emails to users. It’s nothing more than the following:

# backends.py
from django.core.mail.backends.base import BaseEmailBackend
from queue_email.tasks import dispatch_messages

class EmailBackend(BaseEmailBackend):
    def send_messages(self, email_messages):
        if not email_messages:
            return 0
        dispatch_messages(email_messages)
        return len(email_messages)

# tasks.py
rom django.conf import settings
from django.core.exceptions import ImproperlyConfigured
from django.utils.module_loading import import_string
from huey.contrib.djhuey import task


@task()
def dispatch_messages(email_messages):
    try:
        backend = settings.QUEUE_EMAIL_BACKEND
    except AttributeError:
        raise ImproperlyConfigured(
            'No email backend found. Please set ``QUEUE_EMAIL_BACKEND`` value in settings.py'
        )
    try:
        QueueEmailBackend = import_string(backend)
    except ImportError:
        raise ImproperlyConfigured(
            f'Could not import email backend {backend}. Please check the ``QUEUE_EMAIL_BACKEND`` value in settings.py '
        )
    connection = QueueEmailBackend()
    return connection.send_messages(email_messages)

Below is a minimal view that serves the purpose of re-sending the “email confirmation” email to the user when they request it (i.e “Send me the instructions again” type of thing. In the example below we utilizing django-allauth’s send_email_confirmation() method that at some point calls send_messages() of the custom email backend

@login_required
def resend_email_verification(request):
    email_address = EmailAddress.objects.get(user=request.user, primary=True)

    if not email_address.verified:
        send_email_confirmation(request, request.user)

    return render(request, 'accounts/verification_sent.html')

The error we’re facing is:

PicklingError: Can't pickle <class 'django.core.mail.message.Alternative'>: attribute lookup Alternative on django.core.mail.message failed
  File "django/core/handlers/exception.py", line 42, in inner
    response = await get_response(request)
  File "django/core/handlers/base.py", line 253, in _get_response_async
    response = await wrapped_callback(
  File "contextlib.py", line 81, in inner
    return func(*args, **kwds)
  File "django/contrib/auth/decorators.py", line 59, in _view_wrapper
    return view_func(request, *args, **kwargs)
  File "move/allauth_views.py", line 32, in resend_email_verification
    send_email_confirmation(request, request.user)
  File "allauth/account/utils.py", line 284, in send_email_confirmation
    return flows.email_verification.send_verification_email(
  File "allauth/account/internal/flows/email_verification.py", line 208, in send_verification_email
    email_address.send_confirmation(request, signup=signup)
  File "allauth/account/models.py", line 107, in send_confirmation
    confirmation.send(request, signup=signup)
  File "allauth/account/models.py", line 135, in send
    get_adapter().send_confirmation_mail(request, self, signup)
  File "allauth/account/adapter.py", line 637, in send_confirmation_mail
    self.send_mail(email_template, emailconfirmation.email_address.email, ctx)
  File "allauth/account/adapter.py", line 201, in send_mail
    msg.send()
  File "django/core/mail/message.py", line 307, in send
    return self.get_connection(fail_silently).send_messages([self])
  File "queue_email/backends.py", line 10, in send_messages
    dispatch_messages(email_messages)
  File "huey/api.py", line 891, in __call__
    return self.huey.enqueue(self.s(*args, **kwargs))
  File "huey/api.py", line 311, in enqueue
    self.storage.enqueue(self.serialize_task(task), task.priority)
  File "huey/api.py", line 297, in serialize_task
    return self.serializer.serialize(message)
  File "huey/serializer.py", line 76, in serialize
    data = self._serialize(data)
  File "huey/serializer.py", line 70, in _serialize
    return pickle.dumps(data, self.pickle_protocol)

Looks like django now wraps HTML alternatives using a new (?) class Alternative that is no longer defined at the top level of django.core.mail.message so the pickling machinery can’t pick it up.

I guess the way out of this is pass some sort of serialized version of the emails to the email sending Huey tasks where we reconstruct EmailMessage objects but that sounds error-prone and not very DRY.

What are your thoughts?

2

That’s a new bug in 5.2: #36309 (EmailAlternative is not serializable) – Django.

Fwiw, django-celery-email does exactly that, serializing to JSON by default. (In part because of some past security concerns with pickling objects that contain user supplied content.) It’s isolated to your custom backend and task handler, so is DRY enough.