Switch to Playwright tests for integration testing

Feature Request Link:
Use Playwright for integration testing · Issue #13 · django/new-features · GitHub

Project Description:

This project aims to modernize Django’s integration testing by introducing Playwright as an alternative to Selenium. The work includes integrating Playwright with Django’s test framework, migrating existing browser-based tests, and ensuring compatibility with Django’s CI infrastructure. The goal is to improve test reliability, performance, and developer experience for Django core contributors.

Attaching draft PR created in my fork: ​https://github.com/varunkasyap/django/pull/7/

Trac ticket: #37154 (Switch to Playwright tests for integration testing) – Django

My plan is to update Django’s browser testing setup by moving from
Selenium to Playwright

Why Playwright is Better
● Playwright doesn’t need external WebDrivers
● Playwright has built-in auto-waiting for actions.
● Playwright is much simpler to write and read, lowering the cognitive load for
contributors.
● We can delete custom helper functions that we needed for Selenium.

Project Size: As of today, there are 106 selenium test cases in main branch that need to be converted.

Project Goals:

  1. Modernize the testing infrastructure.
  2. Rewrite existing tests
  3. Boost CI execution performance.
  4. Enhance test reliability.
  5. Improve the developer experience with better tools

The project preserves Django’s testing architecture and only replaces the browser
automation layer (Selenium to Playwright).

8 Likes

Thanks for posting @varunkasyap. This is an exciting project :star_struck:

2 Likes

This would indeed be very beneficial. However, the devil is in the details and it might take a considerable amount of time to rewrite all the tests.

1 Like

Great idea! :ok_hand: Agree with Jacob about the effort, though.

Summary of progress:

  • Created base classes: PlaywrightTestCase,
    AdminPlaywrightTestCase, and AdminWidgetPlaywrightTestCase.
  • Deleted one liner helpers that is not really needed in Playwright, example: select_option(), select_option(), is_disabled()
  • Added the --playwright flag in tests/runtests.py
  • Added Playwright to the test requirements and set up the CI workflows
  • Translated 55 tests out of roughly 106 :tada: half way
    Draft PR: https://github.com/varunkasyap/django/pull/7/

Design Discussion:

Even though the translated tests use self.assertEqual(), some waiting is still needed, such as wait_for_load_state() or wait_for(), because a few tests are flaky.

I am leaning towards using Playwright’s expect() since it provides built-in auto-waiting and retrying. Also we could use inbuilt helpers in expect(). However, I don’t think expect() is needed for every test case. For example, tests without page navigation or tests that only verify backend/database changes can continue using self.assertEqual().

My concern is that having both approaches in the same test file may not be contributor friendly. I am also slightly concerned about importing expect in every test case (from playwright.sync_api import expect).

Another option could be to import expect at the top level, (this could be improved in the future with Python 3.15’s lazy imports) but I’m not sure whether ImportError handling is necessary, considering Playwright is already listed in our requirements and expect is available in playwright.sync_api.

1 Like

Nice progress! You’ve outlined the reasons to use or not use expect(), and I don’t expect that to be too confusing for contributors. About the imports, what about consolidating them in one place in an instance method on the base class? Then test methods would call self.expect().

1 Like

When translating admin_views.test_skip_link_to_content.SeleniumTests, I have realized Tab button works differently in WebKit.

Safari User Guide says: Generic Tab button do not highlight clickable items. Where as Option + Tab does.

Action Keyboard Shortcut
Highlight the next field or pop-up menu on a web page Tab
Highlight the next field, pop-up menu, or clickable item (such as a link) on a web page Option + Tab

Reference: Safari documentation

So, the test is failing in WebKit browser. May be we could either tweak the test or can skip test on WebKit.

In Selenium, the UI elements are mostly located by class, id, CSS, or CSS selector. But Playwright suggests using built-in API locators instead.

Example:

self.page.locator('input[value="Save"]')

should be

self.page.get_by_role("button", name="Save")

Playwright locators documentation

Playwright locators look like:

Why use such web-first locators?

  • Resistance to future UI logic changes (less flaky)
  • It feels like accessibility verification as well.
  • Better test readability

But,

  • Needs rework on almost every test.
  • Language translation issues may arise.

Test: admin_views.tests.SeleniumTests.test_child_popup_not_closed_when_parent_minimized

This Selenium test uses selenium.minimize_window() to minimize the browser window, whereas Playwright does not seem to have an API to minimize the browser.

We could use a CDP session for this, but that would be a Chromium-only solution, and we would need to skip this test on non-Chromium browsers.

Tests affected: 1

For the web first locators, I would suggest that this is out of scope for the migration
We could chose to rework tests that do not involve testing translations, however tests decorated with @screenshot_cases(["rtl"]) are testing a translated page and we add this decorator to existing tests fairly frequently. Therefore, it is nice if the tests remain translation friendly

I agree web first locators test accessibility better but we can do any updates here aligned with accessibility testing tickets

2 Likes

Created draft PR: Fixed #37154 -- Switched from Selenium to Playwright for integration testing. by varunkasyap · Pull Request #21596 · django/django · GitHub

Feedbacks are really appreciated.