Automate the screenshot generation process for docs - #35830

Continuing the discussion from this comment: GSoC 25 Proposal Draft: Automate Processes Within Django Contribution Workflow - #11 by marcorichetta

I’ll use this topic as a backlog of the process of contributing to ticket #35380

Problem

  • The Django tutorial currently has screenshots that are often outdated.
  • There is no step-by-step process for generating them. It must be done manually like in @sarahboyce’s PR #18290

Proposal

  • Automate this series of steps so that they can be executed with a command - make generate-screenshots or something similar.
  • WIP PR #19514

Suggestions

  • Tim Graham suggests that running the process only on major releases would be sufficient.

Questions

  • Where should the news/ folder be located?
  • Can we compress images as part of the process?
  • How can I run this process on demand?

Todo

  • Fix failing errors (./runtests.py --selenium=firefox,chrome --headless --screenshots)
  • Fix suggestions per PR #18211
1 Like
  1. Debugging selenium tests
  • Pass --pdb and parallel=1
  • Also -k for specifying a test
  • ./runtests.py --selenium=chrome --screenshots --pdb --parallel=1 -k test_flatpages_fieldsets

  1. I see that tests under news/ override take_screenshot from SeleniumTestCase
    I’m not sure what to do about that.

  2. I think I should create another PR because I can’t use #18211.

  3. My screenshots are generated using dark mode, inherited from my OS settings. How to avoid this?

Another approach could be to not take screenshots, but grab the generated HTML, and insert that into the docs. We do this with the iommi docs and it has some nice benefits. I have a short writeup of it here: Don’t automate screenshots, automate iframes | En kodare

Based on Carlton’s django-sphinx-view, I modified that package and added the possibility to integrate Django views directly into the documentation. This allows users to interactively test the views rather than relying on a static screenshot. It also is safer for developers, because the shown code snippets are actually executed in that environment.

This for instance is a representation of a working slug input field using those special Sphinx directives:

And this is my fork of Carlton’s repository:

Btw: In Playwright, creating screenshots is really easy and could be automated.

@boxed I read the blogpost, and the approach sounds interesting. In the rendered page, how do you handle requiring a given CSS or other static resources to properly render the embedded HTML?

This link does not work, and I can’t find the main branch in your fork to further check, is there any chance that your fork needs syncing with Django’s?

Could you provide the fragment of the code you are referring to?

Yes, makes sense.

That’s strange because the existing selenium tests define the mode to be used, see (for example) the decorator @screenshot_cases(["desktop_size", "mobile_size", "dark", "high_contrast"]).

We configure the style for documentation generation to use absolute URLs for the CSS and JS. It’s quite straight forward and doesn’t affect the samples in our case, but might need a regex replace o something for Django docs, depending on the situation.

Thanks, now it works :white_check_mark:

My first question is why take_screenshot is being overwritten.
This raises a second question: what’s the purpose of the /news app?


2. Dark mode

It’s weird to me because the only input to the decorator comes from SIZES = ["docs_size"].

I ran ./runtests.py --selenium=chrome,firefox --screenshots -k test_user_actions and both Chrome and Firefox are in dark mode.

I’m using EndeavourOS (Arch based)

take_screenshot has been overriden to ensure two things:

  1. the screenshot is saved in the adequate folder (the docs images3. )
  2. the file has the right name, so we can update the images in the docs via running the test (i.e., no further manual action is needed)

This app is necessary to generate the same screenshots that are used in the docs. If you take a look at Admin actions | Django documentation | Django you see that the screenshot use this “news” app with a specific set of values. To automate the generation of these, we depend on having the proper setup.

About the dark mode, I’m not sure what is going one, but perhaps we could progress the tasks as a whole and we debug this later on?

Hopefully this helps! Thank you for working on this!

Hi @marcorichetta,

I played around with nessita’s PR some time ago. Since you’re taking their PR forward, I am sharing my thoughts if it helps.

Where should the news/ folder be located?

This being part of doc-related tooling but not the actual tests, I think it’d make sense moving it to somewhere in docs/ (docs/_ext or a new docs/_extras?).

My screenshots are generated using dark mode, inherited from my OS settings. How to avoid this?

Environment-related inconsistencies are common with test suite. See: Update `Running the Selenium tests` documentation to link to django-docker-box · Issue #40 · django/new-features · GitHub.
Alternatively: inject localStorage.setItem('theme', 'light'); to force light mode.

Two more points I want to add.

First, I think model admins should be registered on custom admin site instead of the default one, as this is also how the rest of test suite is organized. Also see AdminSite.get_app_list() doesn't seem to respect TransactionTestCase.available_apps for why this might be needed.

Second, regarding the misplaced <select> dropdown in screenshots: I think Selenium struggles to capture for the same reason clued here Screenshots do not include expanded <select> · Issue #1306 · puppeteer/puppeteer · GitHub.
Possible workaround is to inject some CSS for Customizable Select Element (Explainer) | Open UI which recently got shipped in Chrome 135. Chrome’s implementation of customizable select puts the dropdown layer under a shadow root making it appear on DOM so it gets picked up by Selenium correctly (tested on Windows).
Note that when testing on Firefox on Windows, the dropdown doesn’t appear at all in screenshots, with or without customizable select.