django.test.TestCase raising exception TransactionManagementError and MySQLdb.OperationalError

amirjj · April 6, 2024, 11:42am

I use django.test.TestCase for my project testing, however altough I’ve read django topic documentation below part is ambigouse for me. below two method for writing LoginTest gets two types of errors:

from django.test import TestCase


class LoginTest(TestCase):

    @classmethod
    def setUpClass(cls):
        super().setUpClass()
        User.objects.create_user(username='my_test_user', password='123456')

    @classmethod
    def tearDownClass(cls):
        super().tearDownClass()

    def test_index(self):
        response = self.client.get('/')
        self.assertRedirects(response, "/accounts/login/?next=/")

    def test_login(self):
        response = self.client.post('/user/login/', {
            'username': 'my_test_user',
            'password': '123456'
        })
        self.assertEqual(response.status_code, 302)

for ebove code I get below MySQLdb.OperationalError:


$ python manage.py test
Found 2 test(s).
Creating test database for alias 'default'...
System check identified some issues:                                                                   
                                                                                                       
WARNINGS:                                                                                              
config.Expert.full_name: (mysql.W003) MySQL may not allow unique CharFields to have a max_length > 255.
        HINT: See: https://docs.djangoproject.com/en/4.2/ref/databases/#mysql-character-fields         
                                                                                                       
System check identified 1 issue (0 silenced).                                                          
.E
======================================================================                                                                                        
ERROR: test_login (tests.test_routes.test_login.LoginTest.test_login)                                                                                         
----------------------------------------------------------------------                                                                                        
Traceback (most recent call last):                                                                                                                            
  File "C:\code\nasim project\venv\Lib\site-packages\django\db\models\sql\compiler.py", line 1562, in execute_sql                                             
    cursor.execute(sql, params)                                                                                                                               
  File "C:\code\nasim project\venv\Lib\site-packages\django\db\backends\utils.py", line 67, in execute                                                        
    return self._execute_with_wrappers(                                                                                                                       
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                       
  File "C:\code\nasim project\venv\Lib\site-packages\django\db\backends\utils.py", line 80, in _execute_with_wrappers                                         
    return executor(sql, params, many, context)                                                                                                               
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                               
  File "C:\code\nasim project\venv\Lib\site-packages\django\db\backends\utils.py", line 83, in _execute                                                       
    self.db.validate_no_broken_transaction()                                                                                                                  
  File "C:\code\nasim project\venv\Lib\site-packages\django\db\backends\base\base.py", line 531, in validate_no_broken_transaction                            
    raise TransactionManagementError(                                                                                                                         
django.db.transaction.TransactionManagementError: An error occurred in the current transaction. You can't execute queries until the end of the 'atomic' block.
                                                                                                                                                              
During handling of the above exception, another exception occurred:                                                                                           
                                                                                                                                                              
Traceback (most recent call last):                                                                                                                            
  File "C:\code\nasim project\nasim\tests\test_routes\test_login.py", line 21, in test_login                                                                  
    response = self.client.post('/user/login/', {                                                                                                             
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                             
  File "C:\code\nasim project\venv\Lib\site-packages\django\test\client.py", line 948, in post                                                                
    response = super().post(                                                                                                                                  
               ^^^^^^^^^^^^^
  File "C:\code\nasim project\venv\Lib\site-packages\django\test\client.py", line 482, in post
    return self.generic(
           ^^^^^^^^^^^^^
  File "C:\code\nasim project\venv\Lib\site-packages\django\test\client.py", line 609, in generic
    return self.request(**r)
           ^^^^^^^^^^^^^^^^^
  File "C:\code\nasim project\venv\Lib\site-packages\django\test\client.py", line 886, in request
    response = self.handler(environ)
               ^^^^^^^^^^^^^^^^^^^^^
  File "C:\code\nasim project\venv\Lib\site-packages\django\test\client.py", line 176, in __call__
    response = self.get_response(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\code\nasim project\venv\Lib\site-packages\django\core\handlers\base.py", line 140, in get_response
    response = self._middleware_chain(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\code\nasim project\venv\Lib\site-packages\django\core\handlers\exception.py", line 57, in inner
    response = response_for_exception(request, exc)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\code\nasim project\venv\Lib\site-packages\django\core\handlers\exception.py", line 140, in response_for_exception
    response = handle_uncaught_exception(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\code\nasim project\venv\Lib\site-packages\django\core\handlers\exception.py", line 185, in handle_uncaught_exception
    return callback(request)
           ^^^^^^^^^^^^^^^^^
  File "C:\code\nasim project\nasim\core\views.py", line 49, in handler500
    response = render(request, "error/500.html", {})
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\code\nasim project\venv\Lib\site-packages\django\shortcuts.py", line 24, in render
    content = loader.render_to_string(template_name, context, request, using=using)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\code\nasim project\venv\Lib\site-packages\django\template\loader.py", line 62, in render_to_string
    return template.render(context, request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\code\nasim project\venv\Lib\site-packages\django\template\backends\django.py", line 61, in render
    return self.template.render(context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\code\nasim project\venv\Lib\site-packages\django\template\base.py", line 173, in render
    with context.bind_template(self):
  File "C:\Users\Info\AppData\Local\Programs\Python\Python312\Lib\contextlib.py", line 137, in __enter__
    return next(self.gen)
           ^^^^^^^^^^^^^^
  File "C:\code\nasim project\venv\Lib\site-packages\django\template\context.py", line 254, in bind_template
    updates.update(processor(self.request))
                   ^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\code\nasim project\nasim\util\context.py", line 48, in template_context_processor
    context['logo'] = Resource.objects.filter(name="logo").first()
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\code\nasim project\venv\Lib\site-packages\django\db\models\query.py", line 1057, in first
    for obj in queryset[:1]:
  File "C:\code\nasim project\venv\Lib\site-packages\django\db\models\query.py", line 398, in __iter__
    self._fetch_all()
  File "C:\code\nasim project\venv\Lib\site-packages\django\db\models\query.py", line 1881, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\code\nasim project\venv\Lib\site-packages\django\db\models\query.py", line 91, in __iter__
    results = compiler.execute_sql(
              ^^^^^^^^^^^^^^^^^^^^^
  File "C:\code\nasim project\venv\Lib\site-packages\django\db\models\sql\compiler.py", line 1565, in execute_sql
    cursor.close()
  File "C:\code\nasim project\venv\Lib\site-packages\MySQLdb\cursors.py", line 103, in close
    self._discard()
  File "C:\code\nasim project\venv\Lib\site-packages\MySQLdb\cursors.py", line 95, in _discard
    while con.next_result() == 0:  # -1 means no more data.
          ^^^^^^^^^^^^^^^^^
MySQLdb.OperationalError: (2006, '')

----------------------------------------------------------------------
Ran 2 tests in 0.605s

however when I change the `setUpClass()` to `setUp()`

class LoginTest(TestCase):

    def setUp(self):
        User.objects.create_user(username='my_test_user', password='123456')

    def test_index(self):
        response = self.client.get('/')
        self.assertRedirects(response, "/accounts/login/?next=/")

    def test_login(self):
        response = self.client.post('/user/login/', {
            'username': 'my_test_user',
            'password': '123456'
        })
        self.assertEqual(response.status_code, 302)

I get below TransactionManagementError


======================================================================                                        
ERROR: test_login (tests.test_routes.test_login.LoginTest.test_login)                                         
----------------------------------------------------------------------                                        
Traceback (most recent call last):                                                                            
  File "C:\code\nasim project\nasim\tests\test_routes\test_login.py", line 8, in setUp                        
    User.objects.create_user(username='my_test_user', password='123456')                                      
  File "C:\code\nasim project\venv\Lib\site-packages\django\contrib\auth\models.py", line 161, in create_user 
    return self._create_user(username, email, password, **extra_fields)                                       
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                       
  File "C:\code\nasim project\venv\Lib\site-packages\django\contrib\auth\models.py", line 155, in _create_user
    user.save(using=self._db)                                                                                 
  File "C:\code\nasim project\venv\Lib\site-packages\django\contrib\auth\base_user.py", line 76, in save      
    super().save(*args, **kwargs)                                                                             
  File "C:\code\nasim project\venv\Lib\site-packages\django\db\models\base.py", line 814, in save             
    self.save_base(                                                                                           
  File "C:\code\nasim project\venv\Lib\site-packages\django\db\models\base.py", line 877, in save_base        
    updated = self._save_table(                                                                               
              ^^^^^^^^^^^^^^^^^                                                                               
  File "C:\code\nasim project\venv\Lib\site-packages\django\db\models\base.py", line 1020, in _save_table     
    results = self._do_insert(                                                                                
              ^^^^^^^^^^^^^^^^                                                                                
  File "C:\code\nasim project\venv\Lib\site-packages\django\db\models\base.py", line 1061, in _do_insert      
    return manager._insert(                                                                                   
           ^^^^^^^^^^^^^^^^
  File "C:\code\nasim project\venv\Lib\site-packages\django\db\models\manager.py", line 87, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\code\nasim project\venv\Lib\site-packages\django\db\models\query.py", line 1805, in _insert
    return query.get_compiler(using=using).execute_sql(returning_fields)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\code\nasim project\venv\Lib\site-packages\django\db\models\sql\compiler.py", line 1822, in execute_sql
    cursor.execute(sql, params)
  File "C:\code\nasim project\venv\Lib\site-packages\django\db\backends\utils.py", line 67, in execute
    return self._execute_with_wrappers(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\code\nasim project\venv\Lib\site-packages\django\db\backends\utils.py", line 80, in _execute_with_wrappers
    return executor(sql, params, many, context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\code\nasim project\venv\Lib\site-packages\django\db\backends\utils.py", line 83, in _execute
    self.db.validate_no_broken_transaction()
  File "C:\code\nasim project\venv\Lib\site-packages\django\db\backends\base\base.py", line 531, in validate_no_broken_transaction
    raise TransactionManagementError(
django.db.transaction.TransactionManagementError: An error occurred in the current transaction. You can't execute queries until the end of the 'atomic' block.

----------------------------------------------------------------------
Ran 2 tests in 1.314s

FAILED (errors=1)
Destroying test database for alias 'default'...

I know setUp() method will run before each test_* method but what is the root cause of raising TransactionalError? does test_* run parallel I know not!

also as I tested if this LoginTest class have a single test_* method everything is fine.

is there any article to explain the order and mechanism of running internal django.test.TestCase function?

amirjj · April 6, 2024, 1:15pm

inhereting from TransactionTestCase instead of TestCase solved my problem.

there is a part in django documentation says:

`TransactionTestCase` ¶

class TransactionTestCase ¶

TransactionTestCase inherits from SimpleTestCase to add some database-specific features:

Resetting the database to a known state at the beginning of each test to ease testing and using the ORM.

does it mean we can’t use ORM in a test case inhereted from django.test.TestCase and for those cases we should use TransactionTestCase?
it seems so becouse in my code I just used User.objects.create_user() in just setUp() with no other ORM and I got TransactionError

but below is Django internal code, the main question is what are those situations that useing TransactionTestCase is preffered to TestCase?

class TestCase(TransactionTestCase):
    """
    Similar to TransactionTestCase, but use `transaction.atomic()` to achieve
    test isolation.

    In most situations, TestCase should be preferred to TransactionTestCase as
    it allows faster execution. However, there are some situations where using
    TransactionTestCase might be necessary (e.g. testing some transactional
    behavior).

    On database backends with no transaction support, TestCase behaves as
    TransactionTestCase.
    """

In case you want to use more than one transaction in your test case class, you have to use TransactionTestCase (this include if you want to have a transaction in setUp() method as this method run before each test_* method). as documentations says TransactionTestCase reset the database state to a know state after each test.

adamchainz · April 9, 2024, 1:04pm

Your issue may have been caused by creating data in setUpClass(). The setUpTestData hook should instead be used for creating class-level data: Testing tools | Django documentation | Django .

No, TestCase is the one to use for the ORM in 99.9% of cases. It’s also a lot faster: https://adamj.eu/tech/2019/07/15/djangos-test-case-classes-and-a-three-times-speed-up/

When you actually need to use a SQL COMMIT during your tests. This is rare, and you’d probably know you needed it. One example is MySQL’s full text search, which can only search on committed data, making TestCase a bit useless.

My (more advanced) testing book contains some more details on TestCase versus TransactionTestCase: Speed Up Your Django Tests

Topic		Replies	Views
manage.py test app results in django.db.utils.OperationalError: no such table: django_site Using Django	2	3856	November 4, 2021
Django tests not flushing the database Forms & APIs	4	481	November 24, 2022
one test failed after testing the suite Django Internals	6	1257	May 5, 2021
Help Needed, Error while running the tests with mysql database Mystery Errors	0	296	May 8, 2023
Test Case on Models - I'm stuck Mystery Errors	2	581	June 2, 2023

django.test.TestCase raising exception TransactionManagementError and MySQLdb.OperationalError

TransactionTestCase ¶

Related Topics

`TransactionTestCase` ¶