Unexpected SynchronousOnlyOperation error

Using Django Async/Channels
1

Hello folks,

I’m having a problem and running out of options. I have an app that I’ve decided to make asynchronous. In this app I have several models and several helper functions that make the ORM queries. In my view I call those functions and create a dashboard for the user.

Since the ORM is synchronous, I decorated those functions with the Django Channels database_sync_to_async decorator. For most of them it works perfectly, but for the two functions that query one specific model (Transaction) it doesn’t work.

Here’s a part of my view:


async def dashboard_view(request):
    # rest of the code ommited
    (balance, transactions, accounts, categories,
     installments) = await asyncio.gather(*[
        functions.get_group_balance(request.user, month),  # all these functions have the decorator
        functions.get_group_actual_transactions(request.user, month),  # the ones that query the Transaction model don't work
        functions.get_group_accounts(request.user, month),
        functions.get_group_categories(request.user, month),
        functions.get_group_installments(request.user, month)  # this one too
        ])

    return render(
        request,
        "transaction/dashboard.html",
        {
            "balance": balance, "transactions": transactions,
            "accounts": accounts, "month": month, "categories": categories,
            "installments": installments, "next_month": next_month,
            "previous_month": previous_month
        }
    )

The helper function:

@database_sync_to_async
def get_group_actual_transactions(user, month):
    group_transactions = # code ommited, but it's a filtered Queryset from a related model

    group_transactions.total_income = group_transactions.filter(
        type='income').aggregate(Sum('amount'))['amount__sum'] or Decimal(0)
    group_transactions.total_expense = group_transactions.filter(
        type='expense').aggregate(Sum('amount'))['amount__sum'] or Decimal(0)
    group_transactions.total_credit_card = group_transactions.filter(
        type='invoice_payment'
    ).aggregate(Sum('amount'))['amount__sum'] or Decimal(0)
    group_transactions.total_savings = group_transactions.filter(
        type='savings').aggregate(Sum('amount'))['amount__sum'] or Decimal(0)
    group_transactions.total_withdrawal = group_transactions.filter(
        type='withdrawal'
    ).aggregate(Sum('amount'))['amount__sum'] or Decimal(0)

    return group_transactions

Template:

{% for transaction in transactions|dictsortreversed:"date"|slice:5 %} # this line raises the error. I've tried removing the filter, thinking it's not compatible with async but it didn't work either

{% for transaction in installment.upcoming_transactions %} # if I comment the other part, then this one raises the error

Error:

SynchronousOnlyOperation at /dashboard

You cannot call this from an async context - use a thread or sync_to_async.

image
image1047×328 9.59 KB

Looking at the traceback, it seems to be raised when the database is queried, but IMO it should work fine because of the database_sync_to_async decorator. And it’s also a mistery why it works with all other functions but not with these two. Most of them return Querysets that I iter through in the template.

I tried commenting out the custom manager for this model and it didn’t work either, whereas other models also have custom managers.

Please help!

2

Keep in mind that a queryset is a lazy object, and is not executed when defined - but executed later. Likewise, foreign key references are not necessarily resolved when the queryset is executed, but when the data is referenced unless you use select_related.

What this means in this case is that you need to ensure that all data is retrieved within your wrapped db function before returning to the caller.

Example:

@database_sync_to_async
def get_some_data(data):
    return MyModel.objects.filter(some_field=data)

This will throw an error when you try to access the data from MyModel, because the function is returning a queryset and not the data from the model.

The safer way of doing this is:

@database_sync_to_async
def get_some_data(data):
    return list(MyModel.objects.filter(some_field=data))

or

@database_sync_to_async
def get_some_data(data):
    return list(MyModel.objects.filter(some_field=data)).values()

or any of the functions that will return anything other than an unevaluated queryset.

Note that aggregate does not return a queryset. See QuerySet API reference | Django documentation | Django

3

Thank you, but most other functions also return unevaluated querysets and they work. Also, when I look at the traceback it shows the transactions I expect:

image
image1322×392 17.4 KB

Also, I tried returning a list as you said and it didn’t work:

image

4

I’m skeptical. You would have to demonstrate that to me.

I’m not sure what value the browser-based display has in this situation.

There’s not enough of your project shown here to perform a full diagnostic. If it’s not working, then you’ve still got an unevaluated queryset somewhere.

5

I got it to work using list comprehension instead of list(): return [transaction for transaction in group_transactions]. But I still don’t quite understand why it didn’t work before.

Here’s an example:

@database_sync_to_async
def get_group_categories(user, month):
    categories = Category.objects.filter(#ommited)
    for category in categories:
        category.total = category.category_transactions.filter(
            #ommited
        ).aggregate(Sum('amount'))['amount__sum'] or Decimal(0)

    return categories

{% for category in categories %} works fine.

6

As pointed out above:

Also note that by:

You are resolving the queryset to access the individual entries.

7

But I do use aggregate here too. Is the problem that I don’t iterate over the Queryset, then?

8

Well, the issue is that you’re not causing the queryset to be evaluated. Iterating over it is just one of the operations that will do that.

See When QuerySets are evaluated for what can cause a queryset to be evaluated.

But yes, you’re creating a QuerySet object with:

Your subsequent operations are creating new queryset objects with the additional filters and then evaluating them with the aggregate function, and assigning those results to new attributes in the original, still-unevaluated queryset.

9

Ok, now I understand. I was cracking my head over that.

Thanks again for your patience and speedy responses! Your help is greatly appreciated.