Mocking ORM to speed up tests?

Using Django
#1

Our tests are getting slow, and if I profile them, I see that a lot of time is spent
inside the database calls.

Rewriting the tests is a lot of effort.

Is there an easy solution like an in-memory ORM?

We use PostgreSQL in development and production.

#2

In-memory ORM’s don’t really work since the Django ORM is really an SQL generator. It would also prevent you from using more advanced features.

You can speed up PostgreSQL by mounting it on an in-memory filesystem. In docker-compose this is as easy as:

...
  services:
    postgres:
      image: postgis/postgis:12-3.0
      ports:
      - 5432:5432
      volumes:
      - type=tmpfs,destination=/var/lib/postgresql/data

The other thing you can do is reduce the amount of database usage in your tests. A key way to achieve this is to use setUpTestData, alongside making sure you use TestCase instead of TransactionTestCase as much as possible.

There’s a lot to cover here… so much I wrote a book… :wink:

4 Likes
#3

Adam’s book is great. IMO, it’s a no brainer to pick up if you’re working at a company that uses Django with an unoptimized test suite. And even if your company’s test suite is pretty optimized, there are some great tips in the book that would likely improve the performance further.

#4

I bought the book and it contains a lot of things which are new to me, although I use django since several years .Thank you for the book.

Maybe you have a solution to this fundamental question:

Reading the the ascii output of pytest daily feels like in stone-age and pytest-html is not much better.

#5

Thank you!

Personally I feel fine with the default pytest output. But I can see how it would be useful.

PyCharm has a graphical test runner that I believe integrates with pytest.

On the terminal, some advanced terminal emulators like iTerm support extra escape sequences which may allow adding enhanced functionality there. I know pytest-travis-fold uses similar escape sequecnes to render folding sections on Travis.