Discuss migrating off of Trac onto 'something' else

[doom]

@thibaudcolas

Personally I’d be surprised if GDPR compliance was a hurdle. I suspect any alternative platform we’d move to would have better tools to manage compliance than we have right now.

I feel like my point is being misunderstood, so I want to clarify. I agree that any SaaS service under consideration will likely be compliant and equipped with solid privacy management tools.

However, my point is about the data itself—specifically, what can and cannot be migrated to the new service. For example, someone who agreed to Trac’s privacy terms must also consent to the terms of the new service. If they don’t, their personal data may not be eligible for migration, regardless of how good the new service’s compliance tools are.

Having hopefully clarified that, I am glad to hear there’s an awareness of GDPR compliance during the migration.

[tom]

I am, as the tradition goes, not a lawyer. However I believe it’s possible to transfer (which counts as processing) this data on the legitimate interest basis.

You could maybe make an argument that it isn’t necessary to have e.g. correct usernames here. I’d argue otherwise, because we’d need to be able to follow up with people that created tickets in the past.

Looking at the balancing rule, I think it’s also clear that this is acceptable, because those people would (as Thibaud says) have better control over their data than they do currently.

I am not sure the terms of service really matters here - unless you can show how? If we can get a way to put these comments under the users’ own (e.g.) GitHub account, well, they already accepted GH’s ToS when they signed up there. And for comments where we don’t know the original user, they aren’t actually using the service in the first place. This might be the only tricky part if they then wanted to remove their data but didn’t want to sign up for GitHub to do so - but we could as a project still do this, and it would be similar to how we would handle this with Trac.

[CommanderStorm]

Hi.

Sorry to necropost, but I think the main issue why NOT to migrate issues to github was structured issue metadata.
This is now in public preview, so the main point why staying on trac is at-trac-tive (sorry) is now resolved…

https://github.com/orgs/community/discussions/189141

Last trac release was 23.09.2023…

[alexgmin]

I think the appetite to move to Github for issues is even less now considering all of the issues and low availability Github is having. And there’s a big concern in the community about being trapped in Github’s ecosystem.

Everybody is aware Trac development has basically stopped, but that doesn’t mean Github is the best path forward, and as one of the people that worked on the informational DEP about this that never got finished, I don’t think there’s enough consensus to do a migration to Github.

[CommanderStorm]

That is fair.

So I am hearing is that a proposal for moving to any platform (codenber, GitHub, ..) are unlikely to be a viable migration path in the next year?

Likely Codenberg is not viable given the current search experience or would that being an open platform (and thus fixable) be a good enough, or? Is this even “on the table”?

Are there issues that can be worked on here, or is this “just currently this way and the appetite for change is not large enough”

[smithdc1]

As a data point ffmpeg are a user of trac see trac.ffmpeg.org.

Based on my reading of their mailing list – until about a year ago they managed commits by mailing list but have now moved to Forgejo.

FFmpeg

As part of that issues were in two places, trac and Forgejo. Where they have currently landed is new issues on Forgejo, but have not yet agreed a plan to migrate data over from trac.

[tom]

I still have appetite to move to GitHub Issues. I think it makes the most sense since we’re already there and it’s what people expect. I think one of the main wins of the new features process is not necessarily that the process in itself is especially good, more that it’s more visible in a place more people are checking, and it’s a little more active because of this. So trying to get everything onto a single platform still seems very useful to me.

The main hurdle to overcome, I think, is that historically the heaviest users of Trac (fellows etc.) are used to it and like using it. And the other important one is the painful migration process.

[carltongibson]

For me, the new features repo is already showing the discoverability problems with GitHub Issues. It’s already at the point where you can’t find the important discussions for all the noise. Beyond those new ones on the first page it’s becoming a black hole.

This has always been the (narrowly technical) problem with GitHub Issues. DRF’s issue tracker (which I worked on for many years) is about a 10th of the size of Django’s and historical issue discovery is a nightmare (and has been for years).

All issue trackers are horrible, all of them, but GitHub’s is so much weaker than Trac that I find it hard to summon the words.

And then there’s everything else GitHub are doing and becoming… IMO it’s approaching the point where we should be discussing leaving the platform entirely, not moving more on to it.

[StephanieAG]

I share concerns regarding GitHub, and I don’t want to use it for my own projects so I’ve recently been looking into alternatives that would work for me which included Taiga and Tenzu which @shaib already mentioned earlier in this thread.

I’m currently working solo on my project which is in its early days, so obviously my considerations when switching are going to be different than Django’s more intense use case. However, since I did the research, I figured I might as well post some of the more relevant notes/thoughts here too.

About Taiga & Tenzu
Adding this in because I was initially confused about the relationship between Taiga, TaigaNext, and Tenzu. For context, the original team that was working on Taiga was Kaleidos. Kaleidos started a full rebuild of Taiga called TaigaNext. Kaleidos wanted to focus on a different open source project of theirs (Penpot, if you’re curious) so they put out a call for others to take over TaigaNext. This resulted in the Kaleidos team transitioning TaigaNext to the Biru team who rebranded TaigaNext as Tenzu. The original Taiga project still exists separately in addition to Tenzu. Both Taiga and Tenzu are still django-based and open source. Quoting from my post on the Tenzu forum:

The TL;DR that I gather from different sources across the two [Taiga and Tenzu] forums and websites is that:

• Tenzu is actively maintained and developed but not accepting outside contributors yet, but has an intention to involve external contributors in the future (I think?)
• Taiga is maintained (I think?) but not in active development (and therefore no longer working with external contributors, I think?)
• over time, it looks like the intention is to have Tenzu naturally surpass Taiga as the default choice between the two once Tenzu covers feature parity with Taiga but with an improved UI/UX/etc. and folks actively working on it

Discussing Taiga and Tenzu as potential alternatives for Django
I wasn’t evaluating Taiga / Tenzu for Django’s specific use case when I was looking at them so keep that in mind, however here’s my take on this after briefly trying them both out for my own small project use.

Being the original older version, Taiga is much more feature-rich, however I don’t think it feels worthwhile to consider it as a Trac replacement: if a primary motivator to consider switching from Trac is the concern about being in maintenance mode, Taiga would be no different in that respect and, given that Tenzu seems meant to replace/improve it and there doesn’t seem to be an immediate urgent need to move off Trac, Taiga feels like a “why bother considering more deeply” option to me.

That leaves Tenzu which, in its current form, would not be a suitable option for Django. It’s very minimal right now, and I don’t even feel comfortable using it for my small project just yet.

However, given that there doesn’t seem to be another alternative that everyone’s already in enthusiastic agreement on and Trac seems to continue to serve the needs in the meantime, I think waiting and keeping an eye on Tenzu’s development would be a good idea.

From a values/operational standpoint, Tenzu clearly aligns:

• Tenzu is open source and django-based, probably making it easier in the future to have more ability/autonomy for directly addressing any problems that arise (but without actually building our own system as was discussed earlier as an option unlikely to be feasible)
• Tenzu is striving to “meet strict accessibility standards,” as discussed somewhat in Tenzu’s Logbook #11
• Biru is a cooperative and is legally registered as a SCOP in France (see the “Our Structure” section on their website for more on what that means) which would make it much more likely for Tenzu to indefinitely remain open source, independent, etc.
• Tenzu’s hosted version has a voluntary pricing model, which would provide flexibility in whether or not to self-host as well as, if using their hosted version, the flexibility to decide the amount to voluntarily contribute and change that amount even to $0 if, for example, the DSF had a bad year of donations due to an economic shift or something

My personal hesitation around Tenzu is around their financial sustainability and my uncertainty about their longevity, specifically whether the Biru team has enough cash runway to make the product into something valuable enough that Tenzu becomes financially stable on its own before their runway runs out. I posted a question about this on the Tenzu forum to see if they might be willing to share more information about it.

I also took the liberty of posting a link to this thread on Tenzu’s forum in case they’re interested in reading what folks here value in a product like theirs.

[Chadys]

Wow, wasn’t expecting Tenzu to get mentioned here ^^

I’m one of the developer behind Tenzu, I think @StephanieAG summarised it pretty well (and I answered her question about our financial state) but feel free to ask me any additional questions.

I can provide some more details about our roadmap which might be relevant here:

• We are currently working on adding tags and and custom fields to enrich the tickets, as well as a search and filters system
• As Stephanie said, we are very much committed on accessibility, we have work planned on this before the end of the year in order to make Tenzu much better and we will follow this with an accessibility audit, thanks to the grant we received from NlNet
• We are planning to work on integrating with forges (gitlab, github, forgejo) but I can’t give any date
• We are planning to engineer a plugin system in order to enable anybody to develop something for their specific use case (like, an auto triage banner display for example , completely random example of course ). But once again, no date given
• Today, we don’t have any way to make a project “public”, either in read-only or otherwise. You need an account and to have been invited to collaborate on the project. That doesn’t mean it won’t come one day (and it was planned in Taiga Next initially), but this is not part of our current roadmap

Not sure if Tenzu could be a good fit for Django trac replacement, it definitely look like it would need a fair bit a tweaking to support the django workflow and the current version is certainly not sufficient for it.

It is a huge endeavour and it’s good this is considered in advance, before the issue become any more pressing if trac goes completely unmaintained. As a django community member, I’m also not a fan of the github solution, because of what stand behind github. I would love for the issue tracker to keep being an opensource tool, no matter which one is chosen.

[tom]

Could you perhaps elucidate? Otherwise we’re going to find it difficult to evaluate any alternative.

What features specifically does Trac have that you would consider another tracker without those features to be a non-starter?

Which features are lacking in GitHub Issues that we should make sure any other tracker does have before considering them?

[carltongibson]

I already answered this above, Discuss migrating off of Trac onto 'something' else - #11 by carltongibson, two years ago now. The linked Django-developers thread has a lot in it, and any search of the history will find a whole host of threads that go over similar ground.

I had understood there was going to be a DEP that (at least) went over the ground here, so it didn’t keep falling on folks to make the same arguments. Did that ever happen?

[tom]

Carlton, forgive me if I’m missing something. You said:

And it can’t become impossible to filter/search. (This last is a problem with GitHub issues. DRF, for example, a large project that I’ve worked on at length, is much harder to find related issues on, despite probably having only a tenth or less of the history.)

This is all well and good. But it doesn’t answer what I asked. I want to know what actual features you feel are needed. “The ability to search and filter” is very broad. You can already search. The filtering story is less great - cpython solved this with many many labels. I personally am not a big fan of this filtering system, but it does technically fulfill your requirement of “filtering”, does it not?

So I’m just asking you to state plainly what you think is needed, because I don’t want to get into the situation where I present something that fulfills the needs you stated (searching and filtering), and then you come up with more needs that you need fulfilling, with the goalposts forever shifting until everyone runs out of spoons to even try to have this conversation.

[tom]

@carltongibson

I had understood there was going to be a DEP that (at least) went over the ground here, so it didn’t keep falling on folks to make the same arguments. Did that ever happen?

The DEP lost steam I think. I do plan to revive it, and this discussion is part of that. I would find it difficult to lay out some proposals without really understanding the requirements. As it stands, I think the requirements (as I understand them) are quite doable, especially with what’s happened in the last handful of years.

[jacobtylerwalls]

On the subject of requirements:

• While GitHub issue templates allow providing picklists to insert metadata into the issue body, I don’t think this syncs to labels. So if our plan is to use labels for metadata, we’d be deferring all the metadata capture we expect to reporters do themselves (component, type, version) to triagers. (Triagers double-check this stuff anyway, but we would be degrading the starting point, and that’s not nothing.)
• We already use labels for CI triggers, so we’d end up with label soup.
• If the Trac timeline goes away, I’ll have to vibe code a replacement via RSS or something
• This is just off hand, happy to do more comments on an informational DEP

I’m trying to keep an open mind here, but I haven’t seen much engagement with Tim’s estimation of the migration cost:

@timgraham

We could try to migrate old issues to a new tracker and add redirects on code.djangoproject.com to prevent broken links, but I suspect any migration to another tracker is going to be imperfect and result in things like broken links, improper formatting, etc.

I believe we would need a very, very compelling reason to migrate to another issue tracker.

We have multiple folks offering to pitch in with Trac maintenance:

@timgraham

I don’t know what level of risk there is of Trac going unmaintained, but I would be inclined to chjp in with maintenance help if needed rather than face the prospect of migrating to another tracker.

@rtpg

I am comfortable volunteering my time to contribute to a one-time “get Trac working on Python 3.13” patch effort

Add me, there’s at least three hands.

Trac already works on recent Python; the stable branch has the classifiers. We are just waiting for a tagged release instead of installing from a stable branch. I pinged the trac list a month ago, and they said they were just landing some last features first (last commit two days ago).

@tom

Python made it work, I think we can too, especially if - as Markus suggests - we ask them for some advice.

I watched this from afar, and it looked like a ton of effort. GitHub staff were involved. Dry runs in test repos. Managing this will distract from developing Django. There’s an optics question there, for me.

Checking my understanding about the DEP, would it be framed as informational/requirements-gathering or a proposal? If it’s a proposal, I saw chatter above about counter DEPs being opened, so I’d like to avoid a proposal DEP at this stage.

[carltongibson]

@tom

… especially with what’s happened in the last handful of years.

For a moment then I thought you would point to a new issue tracker that we didn’t know about. But then I thought you’re probably thinking about all the project features that GitHub added… — I do hope (but not with much confidence) that you’re not.

Re-reading this thread, there are plenty of concrete points that have been made. I haven’t got the energy to go over them again. Not for something I think is a time-sync without benefit.

The bottom line is we have 1000+ open issues. That’s overwhelming for any newcomer, no matter the issue tracker. We have 20+ years of history. That’s overwhelming for any newcomer, no matter the issue tracker. The idea that swapping is going to make a significant difference just isn’t plausible. We’ll do all the work and be in the same boat, and a much worse boat if GitHub is really your target.

FWIW, Trac might be old, but it’s amazingly powerful. It makes navigating those open issues and that history not just possible but really feasible (I would say “easy”, but perhaps “easy as any system could” would pass). Our time and effort is much better spent helping new contributors learn to use it, than trying to replace it in the hope of a miracle cure-all.

(I was quite excited by the recent Tenzu discussion, but that’s forward looking, as it’s not ready yet.)

I’d much rather see community effort go into the (some already agreed, but not implemented) proposals that would actually push Django forward.

@tom

Python made it work,

It’s probably anathema to say this, but it’s strictly worse than the old system, IMO. Every time an old link sends me back to bugs.python.org, I think, Oh yeah, this was better.

I understand you won’t agree with that, or probably any of it, but…

[tom]

@jacobtylerwalls

Checking my understanding about the DEP, would it be framed as informational/requirements-gathering or a proposal? If it’s a proposal, I saw chatter above about counter DEPs being opened, so I’d like to avoid a proposal DEP at this stage.

I will respond in depth (hopefully…) to the rest of your post but I wanted to point out that the reason the DEP fizzled out is partially because there were enough downsides that it was difficult to really get a proposal together that really mitigated them all well enough. I think we’re in a different position today. I have no will to create a civil war over an issue tracker so I have no intent of making a strong proposal until such a time that I’m happy that as many concerns as possible have been addressed and I can easily point to not only a lack of downsides, but also clear upsides and a good sense of what downsides and other side-effects still remain.

So if it’s informational or a real proposal is a bit dependent already on getting a reasonable amount of consensus first - or at the very least a very good understanding of the ground we propose to tread.

[tom]

@carltongibson

haven’t got the energy to go over them again.

Don’t worry, I have energy for the both of us Hopefully you at least will have the energy to read and respond. And if you’ve not, that’s fine, they’re here for anyone else to respond to.

My current understanding of the requirements after going through all of this thread and the mailing list follows:

• Rich per-ticket metadata (e.g. components etc. fields in Trac)
• Precise filtering on said metadata, including negative filtering and combining filters
• Ability for reporters to set this metadata
• Ability for community triagers (not in Django org) to be able to change / act on this metadata
• Avoid misuse of other features (label soup)
• Good, well-featured search*
• Preserve history, and have it all navigable and in the same system without losing timestamps, attribution, etc.
• Self-assignment of issues
• Ability to ensure a ticket doesn’t fall through the cracks and go untriaged
• Ability to cross-reference commits, tickets, PRs
• Activity timeline (RSS would be nice)
• An equivalent of the reports page where can set custom queries for e.g. review queue
• Dashboard (I think debatable - do people really look at this? Needs some analytics insights, but happy to take this as a requirement regardless)
• A way to preserve ticket numbers, make sure we can find our way to tickets that have the old Trac ticket numbers, etc.
• Accessible and usable experience**
• The migration should not be overwhelmingly difficult and fit within our resources

And the non-feature brought up is that it would be preferable to have something open source.

Does this look like a reasonable feature set to you and anyone else who cares to read this? Is anything obviously missing?

* Debatable what good search means, in the mailing list Tim said he often used Google site search and stated this wasn’t possible with GitHub - it is, you can narrow by path e.g. site:github.com/django/django/pulls for example. Trac doesn’t have its own search unless you count adding a filter to description / summary, which I find excruciating in Trac’s UI, which is perhaps why people resort to Google site search.

** While accessibility isn’t much debatable, the usability of a platform is subjective, especially if you’re used to using it. This will be a difficult one to answer.

[carltongibson]

I’m sure it’s something like that. No doubt there’ll be more. I understand your point about “moving goal posts” but I’d phrase it more as the difficulty of providing a spec before implementing… — there’s always more you didn’t think of.

This issue is getting a tick, particularly for the history navigation. Actually having lived in Trac gives you an appreciation for how powerful it is, and how weak GH’s version is by comparison.

Discussing feature lists feels a bit previous, even after two years and 60 posts. We don’t even have a consensus that it’s a good idea to move…

—

Update: Nonetheless, let me add “quick to load” to the requirements. A complex query might take a moment, but Trac is generally quick. In contrast, where I live in Spain — I can’t speak for elsewhere; I’m sure it’s great in San Francisco — GitHub’s modern UI often takes many many seconds to even begin to load, and many many more before the (React?) UI is fully responsive. For all the other reasons to get off of GitHub, the degradation of the user experience is one of the most apparent, and pressing — on the squeaky wheel gets the grease basis.

[jacobtylerwalls]

Thanks for starting the list. A few more:

• Ability for community reviewers (not in Django org) to advance or regress swimlanes e.g. ready for merge, needs improvement
• Permission granularity for community users, supertriagers, and deployment/ops

From the other thread:

• Visibility into backports directly from ticket
• No spammy references from force-pushes
• “Next steps” messaging derived from swimlane status

I’d add:

• No references from spammy forks e.g. django-vulns/open-claw-bot [CRITICAL ADVISORY]

Self-managed is a nice to have that might be seen as begging the question:

• As we speak #contributor-discussion on Discord is advising someone banned from GitHub for opaque reasons to submit their patch over Trac. I guess we can accept patches over the forum if we shut down Trac, but that’s not really its purpose.
• How quickly can we get accessibility improvements deployed on a platform out of our control?