Hi everyone, I’m Prince, a new Django contributor.
While updating the description of my documentation PR, #21935, I encountered a confusing issue with the PR quality checks.
My description used * [x] for checked items. GitHub displayed the checkboxes as checked, but the quality checker still reported missing AI disclosure and an incomplete checklist. After replacing the asterisk markers with hyphens (- [x]), the check passed.
I found the “Bot checks for typo PRs” discussion, but didn’t see this particular formatting issue mentioned.
Is this restriction intentional, or should the checker accept asterisk markers too? If it’s a bug, is there an existing ticket, or would a new Trac ticket and a small patch with tests be appropriate?
Thank you!
This is worth a trac ticket. There’s an accessibility concern about using - instead of * for markdown lists too, but that’s a tangent. The fact that github renders it properly is likely sufficient justification for a change.
Thanks for the guidance! I’ve opened ticket #37346 with the reproduction steps and a link to this discussion. I’m interested in working on the fix and regression tests.
Or, we could double down on checking the checklist verbatim as a way of testing whether agentic contributions are checking all the guidelines. Presumably we have some guidance somewhere that says “fill out the checklist as is”, that is, without “improvements”.
I frequently notice agentic contributions adding “improved” whitespace to the checklist. I like having that as a tell.
For instance, we could adjust the PR quality bot message to say, “tell your agent to avoid altering the checklist; in fact, tell your agent to read the XXX and YYY too” etc
I don’t think this is a place for that battle. We have community members that advocate for asterisk based lists on the basis of accessibility. If someone has an agent configured on that basis, why should we to trip them up here?
Thanks for discussing this. I’ve opened PR Fixed #37346 -- Supported asterisk markers in PR quality checks. by prince889035-sys · Pull Request #21978 · django/django · GitHub for ticket #37346.
The patch accepts both hyphen and asterisk task list markers while preserving the existing AI disclosure and required checklist checks. It includes regression tests for both valid and invalid inputs.
I spoke up because I’m worried the PR bot will become a distracting target for maintenance, that’s all. The asterisks seem orthogonal to me. We can just use them in the checklist itself, as Marijke’s post suggests is better for all users anyway.
The checklist is a point of friction; it’s supposed to make the other party slow down. I don’t want to slow ourselves down making it easier for the other party to go faster.
Do you mean we’re going to spend too many cycles iterating on this bot rather than iterating on the the framework itself?
That could be true. Time will tell for sure. I think right now it’s undergoing a bit of flux because:
- It’s a new-ish piece to our process and like any new process we’re discovering friction points that need sanding down.
- The OSS contribution landscape is still changing with LLMs.
I hope with enough time both will settle out.
Alternatively, we could move this bot into the online community WG domain and have them be responsible for maintaining it rather than the Fellows.
@jacobtylerwalls and I talked about this in the DSF office hours and agreed that this is a valid issue, but we don’t have the bandwidth to invest fixing it. There are bigger issues with the bot and LLM contributions that we want to resolve first. Mainly do we need an agents.md file or consolidating the new contributor bot and the PR validator.
@prince889035-sys unfortunately this means this is currently a wontfix and we won’t be accepting a ticket for it until we have clarity on what the future is for the bot.
Thanks for the clarification and for discussing this. I understand the current priorities and will leave this issue here. I appreciate the guidance and will focus on other Django contributions.