Tying transactions to atomic requests

Using Django Using the ORM
1

Hey,

I’m in the process of deciding how to handle transactions for a project I’m building (reading the docs).

Basically the app is an internal organization tool for currently ~12 users at the most. It’s not a public project, thus any increase in traffic would be rather predictable (e.g. they won’t suddenly hire a 1000 person tomorrow). This is stuff like inventory management and internal processes, nothing is automated (e.g. I don’t have some systems making multiple queries per minutes).

Given that, I am tempted to tie transactions to the requests by default. It’s pretty simple, and performance isn’t really a huge issue here. If there’s huge growth later, I’m thinking I have a few options:

  • New views could be managed on a case by case basis with the @transaction.non_atomic_requests, if say I need to add new views that deal with something likely to generate more request than what I currently anticipate
  • If/when traffic becomes an issue, I could turn off the default ATOMIC_REQUESTS, wrap most views with @transaction.atomic and simply not wrap the most used views and manage those isolated cases more precisely.

Questions:

  1. Are my assumptions above sensible? Or is it naive?
  2. The one issue I’m not sure about is exception handling. As mentioned here “avoid catching exceptions inside atomic”. My understanding is that you don’t want to catch db-related exceptions inside atomic (since Django uses those to determine if the transaction as whole should or not commit). However, if I want to catch say an AttributeError or TypeError within my View, that would still be fine, correct?
  3. If the view is atomic, and the transaction doesn’t complete, how can I control what I return to the user? Either return some error, or (ideally) be able to return something thrue the message framework (which is my default handling right now most of the time)?
2

These are definitely only my opinions. Hopefully others chime in:

  1. Are my assumptions above sensible? Or is it naive?

I prefer to use the defaults and add in transaction.atomic where it’s necessary. It’s more explicit about what’s going on and is probably easier for another Django dev to understand coming to the project fresh.

However, if I want to catch say an AttributeError or TypeError within my View, that would still be fine, correct?

Likely yes. That warning’s core issue is “When exiting an atomic block, Django looks at whether it’s exited normally or with an exception to determine whether to commit or roll back”. If one of those above errors occurs and you need to rollback the transaction, you would have to do it yourself if you caught the exception.

3 If the view is atomic, and the transaction doesn’t complete, how can I control what I return to the user?

If you need to return a message speicifc to the view, I believe you’d have to use this format:

from django.db import IntegrityError, transaction

@transaction.atomic
def viewfunc(request):
    create_parent()

    try:
        with transaction.atomic():
            generate_relationships()
    except IntegrityError:
        handle_exception()

    add_children()

Otherwise there is the handler500 callback function.

1 Like
3

Transactions normally increase throughput of your system, because they minimize writes to disk. So you’ll probably see better performance.

I respectfully disagree - transactions are normally necessary in more places than we think. I usually turn it on per-view transactions by default and rarely find views need it disabling. I wish atomic requests defaulted to true.

1 Like
4

Interesting. I’m curious of which areas folks aren’t using transactions where they should be. I could definitely be susceptible to them.

1 Like
5

Hey guys,

Thanks for the advices. Also good to see that it’s not a universal answer. It somewhat justifies to myself having to ask the question!

After playing with both options some and giving it further thougths, my 2 cents on my own question:

  • Explicit is generally good, I agree this makes things somewhat murkier. That being said, I’m going to consider that how the “A” of ACID database is handled should probably be on any programmer’s mind when working on that type of program. And then this is a built-in Django feature. E.g. I’m not building some obscure meta-class to handle weird corner cases everyone will forget about. So although I prefer explicit notation most of the time, I’m willing to live with this loss in this case.
  • Catching is OK, but then in some cases re-raising might be more appropriate. The downside of this approach is that one may forget he’s working within an atomic block.

It has required some time to sort out a few corner cases, especially using signals, but I think it has been much, much faster to have a working application that ensures atomic transactions with this approach (e.g. atomic by default).

6

Hi,

I have built a Java framework around Domain Driven Design concept that uses SERIALIZABLE isolation level and transactions tied to request/response cycles, just as ATOMIC_REQUESTS = True. PostgreSQL’s MVCC is so good that it should be considered the default approach, thus solving particular perfomance issues separately from the OLTP bulk.

I am evaluating Django to decide the correct way of handling serialization errors, in the context of atomic requests. I would be interested to know how you guys, Django experts, would handle the OperationalError thrown by serialization failures. Basically, details on logik’s third question.

I am looking for options on how to deal with a serialization error, and optionally how to retry the Django request as suggested in PostgreSQL’s documentation (PostgreSQL: Documentation: 15: 13.5. Serialization Failure Handling). I was thinking about implementing a middleware to do this, but would like to use an existing practice before rolling my own.

Thanks for any advice!

Best,
Camille