Improving the flow for documentation changes

In the past few days, the opportunity to run only the documentation pipelines when there are changes to the documentation folder has come up more than once [1].

There have been suggestions on how to deal with this on Mastodon by @jeff and @sabderemane , which could be a starting point to improve the situation.

The advantages would certainly be the speeding up of the pipelines for changes to the documentation only, the reduction of energy required for each change to the documentation.

It also emerged in a thread of the Saturn team of the Dajngonaut Space project, how sometimes, when there are changes to the docs, the tests fail for reasons totally unrelated to the changes themselves.

Before opening an issue or proposing code, I wanted to share this idea here and receive feedback.

If someone wants to take charge of the matter because they think they already know how to solve the problem, they are welcome.

[1] Paolo Melchiorre: "Is there a way to manually or automatically trigg…" - Fosstodon

3 Likes

This would be cool. Yes, I’ve also seen there are some flaky tests that can put a red X on otherwise good PRs.

1 Like

Paolo Melchiorre writes:

The advantages would certainly be the speeding up of the pipelines for
changes to the documentation only, the reduction of energy required
for each change to the documentation.

It also emerged in a thread of the Saturn team of the Dajngonaut Space
project, how sometimes, when there are changes to the docs, the tests
fail for reasons totally unrelated to the changes themselves.

So there are basically two issues here?

  • flaky (code) tests, that randomly fail regardless of the changes
  • separating the build process for the documentation from the rest to
    • have faster builds for documentation only changes
    • prevent doc only changes to fail because of flaky (code) tests?
1 Like

The point here is only tu run the only doc-related pipeline when a doc changes occur, having the same tests flow, but only with conditions.
If you don’t run other pipeline (e.g. Python 3.12 Postgres) typically you skip also the flaky tests which you have still to solve to rub tests for other changes, but this is another issue.

Bazel is made for this problem, but I only have experience using it in a private environment where malicious cache poisoning and the like is not a concern. Bit of a Pandora’s box to propose Bazel for Django’s CI.

Though I have to say that when you have Bazel set up, pushing up a readme change and having a green check after 2 seconds instead of 40 minutes is wonderful.

(EDIT: GH actions do seem good enough here though, especially if commits to main branches still get tests run fully)

1 Like

Does anyone have experience with Weblate? It’s built using Django.

If I’m understanding correctly, we want to trigger workflows in a way that the website test suite doesn’t run for docs changes & visa versa.

I think docs workflows could check the paths. Using an example from a project I’ve got;

on:
  push:
    paths:
      - docs/**

Yes, I’ve used this. It’s a great tool for translations

I tried and failed here :sweat_smile:

We seem to be using the git plugin for Jenkins: Git. This has Polling ignores commits in certain paths and I have updated the “Excluded regions” to docs/* in our job configurations - but this doesn’t seem to have the desired effect :thinking:

In short, I am happy to make this happen but I don’t really know how to achieve it :+1:

1 Like

Updated to docs/** :crossed_fingers: need someone to do a docs change and see if that worked

1 Like

Great. But you changed the Jenkins configuration, but that is not public?
I see a lot of small doc changes in PR list, some of them are quite ready to be merged. We can concentrate in one of the smallest one to be able to merge it ASAP and trigger Jenkins.

Correct, the configuration around the job triggers require a Jenkins account (with some access)
Some stuff related to Jenkins is public, like the ansible playbooks:https://github.com/django/jenkins-ansible but I think not relevant for the current task

I pushed to a docs pr. There’s a queue for running things. Build 28062 for pull-request-focal is a docs only change: pull-requests-focal [Jenkins]


:crossed_fingers:

https://djangoci.com/job/pull-requests-focal/28062/
Looks like it’s running them rather than skipping them :frowning:
Hard to tell if the new configuration is applied or whether it needs a reboot or something

1 Like

I think its java regex, so maybe. docs/.*

1 Like

Updated :+1: I think we had this in a couple before I started, let’s see :crossed_fingers:

2 Likes