tests - signals - fixtures - mixing of levels between apps and project?

Hi!

I have been struggling with this issue with my tests and I think it boils down to a confusion of levels between what an app and project is, and if so it may be something to improve in the django architecture. So here is the issue, which I think many will have experienced:

An app can be reusable and included in different projects. An app has its own tests. These tests are self contained and usually load data in the form of fixtures. Even though tests are self contained, they are always run in the context of a django project with its own settings and other apps installed. Issues arise when using signals from one app to the other. Then whatever app1 test is doing, it may trigger a signal receiver in app2, for which the test may not have initialized any data, making the test to break. Its not that there is a problem with app2, only that its data has not been setup. Since app1 should not know about app2, it doesnt load any fixture for app2’s data.

Tests and apps should be self contained but tests are run in the context of a project with many apps, this is the confusion of levels that I’m speaking of. AFAIK I could create different AppConfigs where signals are connected or not, but I don’t know how to use different AppConfigs when testing different apps. It makes sense that the app that connects receivers to another app knows that it should detach these receivers when running a test for that other app.

Maybe the answer is just: “not run tests for app1 in that project, rather run them in another project separately”. But maybe there is a better way? :slight_smile:

I would say that when one is testing across multiple apps, we’re no longer testing units of code. Now we are testing features, and care less about the implementation of those features. These may be better handled by end-to-end tests.

Having said that, sometimes we have signals to apps that do things that we can’t test (for example, bill a user for a subscription). In that case, mocking out those apps (with due caution!) might be the way to go.

I will say that signals do serve a valuable purpose in complex projects when events need to be broadcast across multiple apps. They do make testing a new challenge, however.

-Jorge

Sure, but that testing across apps is another issue, not the one I’m discussing.
I’m pointing to the fact that perfectly good, existing unit tests start to fail when adding signals, because signals touch other apps which have not been initialized properly in those unit tests.

So, you are right, it starts to become that unit tests are not unit any more and start to affect things in other apps. So my question remains: how do we initialize or disable those apps that get triggered via signals? Or, taking your suggestion: how would you go about mocking an app for a particular test?

Both these questions boil down to: how do you supply a different app config when running tests for an app, in a way that the app doesnt have to know about this, but it is some project setting?

In the standard library, unittest.mock provides mock objects that one can use in tests. One can choose how they behave, what data they return from methods, even what exceptions they raise.

While this forum is extremely useful, a deep dive into use cases really doesn’t work here. Fortunately, other people have already done that work. There are many resources on using mocks in Python, and some are Django specific. Pyvideo.org is an awesome searchable resource for talks on mocks.

I’d suggest starting in those resources to get some ideas to solve your particular problem.

-Jorge

While I appreciate your input about mocks, please try to focus on the question asked. It really seems you are not understanding my concern.

I could set up mocks, I could also load some data to initialize the actual objects. But: I want to do this without touching the test file because what needs to be initialized/mocked is in another app that this app should not have to know about.

I understand there is no real place in django to set this up. This is why I’m posting here.

I think you could tone down your response here. @jlgimeno has taken time to write a thoughtful reply and point you to various resources.

As for your question - generally we recommend against intra-project signals.

There’s a warning at the top of the custom signals docs:

Signals are implicit function calls which make debugging harder. If the sender and receiver of your custom signal are both within your project, you’re better off using an explicit function call.

Generally it’s better to e.g. override save() and explicitly call functions in other apps. With this dependency made explicit, you’ll see it is very hard to test e.g. a model in app A which calls a function in app B without knowing anything about app B, without things like mocks.

If you stick with signals, one advanced option is setting available_apps on your test case classes. You’ll need to make sure you disconnect signal handlers when the corresponding app is removed from INSTALLED_APPS due to available_apps.

Really sorry, I got frustrated. I believe this question belongs in this forum, because I m discussing the relation between signals and tests.

Signals are a nice way to produce events and detach the producer and consumer of these events, and this is needed. But I can see how they make things complicated for testing! The problem is that signals get connected always via the code, even though the testing for a particular app doesnt initialize other apps that it doesnt know about.

An example would be a paypal processing library. Whenever it receives a payment, it will fire a signal and another app would know what to do when it receives a payment. In the tests for the paypal app, we dont have to know about any users of this library, so adding mocking code, or data loading, or changing the available_apps from the paypal tests would not make sense.

The more I think about how to overcome this, the more complicated it becomes and my conclusion is just not to run tests for the paypal library app in the project that uses it.