Django 4 and loading tests for specific markets

Using Django
1

Hey!
We use Django for an online webshop, and want to upgrade to Django 4. However, since upgrading on a test branch, we have encountered a problem, that did not appear before upgrading:

We serve two different markets, lets call them Market A and B. Since these two markets can have different functionalities, our INSTALLED_APPS gets populated like so:

MARKET_SPECIFIC_APPS = {
    MARKET_A: [
        'market_a.apps.MarketAConfig',
        'payment_system_a.apps.PaymentSystemAConfig',
        '...'
    ],
    MARKET_B: [
        'market_b.apps.MarketBConfig',
        'payment_system_b.apps.PaymentSystemBConfig',
        '...'
    ],
}

if MARKET in MARKET_SPECIFIC_APPS:
    # If there is a market-specific app, add it to INSTALLED_APPS
    INSTALLED_APPS += MARKET_SPECIFIC_APPS[MARKET]

However, when running MARKET specific tests for each market, only MARKET_A passes. MARKET_B now throws an error, since upgrading from Django 3.2 to Django 4

ImportError: Failed to import test module: payment_system_b.tests
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/local/lib/python3.9/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/home/circleci/project/payment_system_b/tests.py", line 20, in <module>
    from payment_system_b.models import Payment
  File "/home/circleci/project/payment_system_b/models/__init__.py", line 1, in <module>
    from .contract import Contract
  File "/home/circleci/project/payment_system_b/models/payment.py", line 12, in <module>
    class Contract(BaseContract):
  File "/home/circleci/project/venv/lib/python3.9/site-packages/django/db/models/base.py", line 113, in __new__
    raise RuntimeError(
RuntimeError: Model class payment_system_b.models.payment.Payment doesn't declare an explicit app_label and isn't in an application in INSTALLED_APPS.

Meaning, this test is still run, and the test runner includes it in INSTALLED_APPS even tho it should really not.
If it helps, Payment models on both markets derive from an abstract class called BasePayment

Is there any change from the Django 4 changelog im missing about this? Running these market-specific tests was working fine before upgrading to Django 4, running Django 3.2.8.
We did not change the way we run these tests, or load them:

            . venv/bin/activate
            MARKET=MARKET_A python3 manage.py test --tag market_a --parallel 4 --timing
            MARKET=MARKET_B python3 manage.py test --tag market_b --parallel 4 --timing
2

I think you should share the test in order to have a good answer.

Have you tried to run the tests without --parallel argument. I think is possible that your test is not fully isolated and has some side effect from previous tests.

1 Like
3

Here is more information, and the error traces:

INSTALLED_APPS = [
   ...
   'payments', # THIS INCLUDES `BasePayment` Model
   ....
]


MARKET_SPECIFIC_APPS = {
    MARKET_B: [
        'market_b.apps.MarketBConfig',
        'market_b_payments.apps.MarketBPaymentsConfig'
    ],
    MARKET_A: [
        'market_a.apps.MarketAConfig',
        'market_a_payments.apps.MarketAPaymentsConfig'
    ],
}

if MARKET in MARKET_SPECIFIC_APPS:
    # If there is a market-specific app, add it to INSTALLED_APPS
    INSTALLED_APPS += MARKET_SPECIFIC_APPS[MARKET]

======================================================================
ERROR [0.004s]: market_a.test_redirects (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: market_a.test_redirects
Traceback (most recent call last):
  File "/usr/local/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/local/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/e-commerce/market_a/test_redirects.py", line 5, in <module>
    from market_a.models import AdvertisementIdMapping
  File "/e-commerce/market_a/models.py", line 14, in <module>
    class MigratedMissingData(models.Model):
  File "/e-commerce/venv/lib/python3.9/site-packages/django/db/models/base.py", line 113, in __new__
    raise RuntimeError(
RuntimeError: Model class market_a.models.MigratedMissingData doesn't declare an explicit app_label and isn't in an application in INSTALLED_APPS.


======================================================================
ERROR [0.000s]: market_a_payments.models (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: market_a_payments.models
Traceback (most recent call last):
  File "/usr/local/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/unittest/loader.py", line 470, in _find_test_path
    package = self._get_module_from_name(name)
  File "/usr/local/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/e-commerce/market_a_payments/models/__init__.py", line 1, in <module>
    from .payment import Payment
  File "/e-commerce/market_a_payments/models/payment.py", line 12, in <module>
    class Payment(BasePayment):
  File "/e-commerce/venv/lib/python3.9/site-packages/django/db/models/base.py", line 113, in __new__
    raise RuntimeError(
RuntimeError: Model class market_a_payments.models.payment.Payments doesn't declare an explicit app_label and isn't in an application in INSTALLED_APPS.


======================================================================
ERROR [0.000s]: market_a_payments.tests (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: market_a_payments.tests
Traceback (most recent call last):
  File "/usr/local/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/usr/local/Cellar/python@3.9/3.9.10/Frameworks/Python.framework/Versions/3.9/lib/python3.9/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/e-commerce/market_a_payments/tests.py", line 20, in <module>
    from market_a_payments.models import Payment
  File "/e-commerce/market_a_payments/models/__init__.py", line 1, in <module>
    from .payment import Payment
  File "/e-commerce/market_a_payments/models/payment.py", line 12, in <module>
    class Payment(BasePayment):
  File "/e-commerce/venv/lib/python3.9/site-packages/django/db/models/base.py", line 113, in __new__
    raise RuntimeError(
RuntimeError: Model class market_a_payments.models.payment.Payments doesn't declare an explicit app_label and isn't in an application in INSTALLED_APPS.

all these test sit in an app called market_a_payments
This app should not be discovered when running tests for market b:

MARKET=MARKET_B python3 manage.py test --tag market_b --timing

I can try to share a bit of the test, and where it fails according to the stack:

from urllib.parse import quote
from django.test import TestCase, Client
from core.mocks import Mocks
... other imports

MARKET_A = "market_a"
MARKET_B = "market_b"


def valid_market(market: str):
    return market in [MARKET_A, MARKET_B]

def override_market(market):
    if not valid_market(market):
        raise Exception(f"{market} is not a valid market.")
    return tag(market)

class TestRedirects(TestCase):

    @override_market(MARKET_A)
    def test_landing_page_redirects(self):
        client = Client()
        cases = {
       ....
        }

        for input, dst in cases.items():
            with self.subTest(input):
                response = client.get(input, secure=False, follow=True)
                self.assertRedirects(response, dst, 301, 200)

Line 12 is actually class TestRedirects(TestCase):

Django 3.2.8 worked super fine, so i assume sth in the test runner must have changed from that version to 4, as our code remained unchanged in this test.
Annotating the class with the @override_market(MARKET_A) didnt do anything either.

Test are run on circleci using circleci/python:3.9.7-node-browsers image.
The overall goal here is that this test only gets run/discovered when the specified market gets hit.
so MARKET=MARKET_B python3 manage.py test --tag market_b --timing should not discover or run this test. taking off --parallel as suggested didnt do anything either.

Minimal reproduction here:

4

Thanks :slight_smile: I tried that, but it didnt change. I provides a hopefully more detail answer above. Minimal reporduction here: GitHub - Thorbenl/django4-testrunner: Minimal repo for reproducing a bug

5

I have no idea of what could be the problem, but I have been in your situation with a “doesn’t declare an explicit app_label” error, I had have to try and test all the solutions ins this Stackoverflow thread.
I hope it helps.

6

Im at the end of my knowledge, and will probably just downgrade back to 3.2.11 :slight_smile: As that was working.

The test runner must have changed sth, as it previously never included tests of apps that arent in installed_apps :slight_smile: but now it does. So either:

  • tagging changed?
  • testrunner changed
  • discovery of tests changed