Following the release of Django 5.0, I got familiar with the process of incorporating translations from Transifex into the Django source tree. This process is very involved and cumbersome, requiring the fetching of around 1200 .po files (~110 languages for 12 resources) and their corresponding 1200 .mo files. Then, a manually review for each .po file is required to inspect the changes between the last release and the soon-to-be the new release, with the goal to revert any undesired side-effects such as changes in plural forms, and to ensure that only files with new translations are merged.
This manual process ended up taking more than 10 hours, which not only left me feeling quite drained due to time constraints but also didn’t seem like the most efficient use of a fellow’s time. So I would like to propose two orthogonal things:
Short term: try to add some automation to this process. This would include reducing the scope of the files needing manual inspection by using the Transifex API (to fetch only files that were updated after a given date), and using some sort of po-differ to decide “which files have new translations” (if possible).
Medium to long term: I’ll try to analyze other translations tools. Specifically, there is one open source tool that I’ve used a long time ago and I would like to evaluate whether it would be a good fit for our needs or not.
I wanted to share this with everyone in case there is additional background information about the process that I might be missing. Additionally, I’d like to gather other opinions in case others have already thought about this.
I feel your pain Unfortunately, Transifex doesn’t help us in several ways. I hoped at some time to migrate to Weblate, but the process was stuck, mainly for hosting-related reasons.
In any way I encourage you to make any improvements to that process, it will only be a bonus, I’m sure!
Question about this: is it the main issue blocking the migration to Weblate the fact that they offer a “Gratis Weblate for libre projects” plan which only supports up to 60 target languages?
Or is there something else? Have you reached out to them on behalf on Django (asking so we don’t duplicate efforts)?
If you’d like the DSF board to reach out to Weblate (or any other provider) to negotiate usage of any particular plan, please let us know. It’s no problem for us to consider paid options if they feel like they’d save us time.
Thank you @thibaudcolas! That’s an interesting and appealing offer. I haven’t use weblate before so I’m not sure if the cost of a switch would be worth it. @claudep would you have a stronger opinion?
It’s also worth mentioning that since this post, I built and landed some improvements to the “fetch translations script” which makes the translations update process faster and more bearable, so there is no urgent need in this front to make immediate changes.
Right now, I don’t have time and/or energy to invest to lead such a move, however I think it would be beneficial for the project to migrate at some point. So if anyone volunteers, I would made my best to at least help for the transition.
I don’t unfortunately. I clicked around a bit and I could sign in with github easy which was good, and I could contribute, but it seems to suggest to translate things that are already translated in their UI and the UI looks in general more like “look at these fancy graphs/tables!” instead of “here is where we need help”. It also doesn’t seem to show any useful context that I can tell? App Manager/Main — Swedish @ Hosted Weblate: Grant URI permissions for example.
Let’s find someone who knows both Weblate and Transifex and could help us compare? Even if we didn’t end up switching to Weblate, it’d be interesting to compare options and see whether other projects have better integrations with one or the other.
For what it’s worth, looks like Weblate recently became a company, which feels like good news for their sustainability? And their Libre gratis plan is now “160k strings”. I count 93717 msgstr in Django so that should be good.
From my side I’ll reach out to Wagtail folks, we’re at a similar scale with 1000 .po files, 39 languages, fetching translations from Transifex 4 times per year. Automation is a bit different as we use a much simpler fetching script compared to that of Django, but I don’t have the knowledge to understand why.
Weblate has native integration with GitHub with commits direct to master or with pull requests, so it is much easier to maintain. Plus Weblate is GPL and based on Django.
And their Libre gratis plan is now “160k strings”. I count 93717 msgstr in Django so that should be good.
We probably should consider moving docs translations (at least?) along the core Django translations. With them we would definitely go over the standard open-source plan limit.
Unfortunately, Transifex doesn’t help us in several ways. I hoped at some time to migrate to Weblate, but the process was stuck, mainly for hosting-related reasons.