Implementing a Formal Experimental API for Django

[prafulgulani]

Hello everyone,

I’ll be working on the project “Implementing a Formal Experimental API Framework for Django” along with @nanorepublica.

This project aims to create a safe space to experiment with features. Users can opt-in to test these and community can iterate on API designs based on actual feedback.

For DEP 2 rewrite and implementation, I am looking into how we can safely isolate experimental features.

So, beyond namespacing and settings-based flags, are there other methods we should consider?

Is an installation barrier (using package extras) like pip install django[experimental] a necessary safety measure to protect stable releases, or does it introduce too much friction for developers who want to test new features?

[CodenameTim]

Thanks for kicking this off Praful!

@prafulgulani

How should we decide when an experiment ends? The current idea in the proposal is a flexible Evaluation Period where features are reviewed after each minor release, rather than setting a strict deadline. What should be the deciding factors for a feature to be promoted to stable or purged to prevent experimental bloat?

I don’t think we’ll arrive at a standard set of rules for every possible situation. I suspect each feature will have its own reasons for being added in an experimental state, and therefore it would have its own reasons for when it would exit experimental. I think it’s fair for the community to ask for a feature to include why something qualifies as experimental, what are the criteria that needs to be met for it to be made stable and what are the criteria that needs to be met for it to be removed entirely.

One thing that could help those discussions along are guidelines for what are reasonable justifications for experiments and what are some example/plausible exit criteria.

[prafulgulani]

Okay, that makes sense, letting the feature proposer justify the reasons for why the feature needs to be experimental along with the promotion/purge criteria will make the process flexible for different situations.

Since a big part of this project is integrating the experimental framework into Django’s existing contribution process, I was looking at the django/new-features repo for inspiration. They scale their requirements based on feature size (Days to Weeks, Months and Quarters). We could use that same sizing logic to provide the guidelines and examples.

Ultimately, these guidelines will depend a bit on how the experimental API framework is implemented. So, do you have any ideas on how that should be implemented?

[CodenameTim]

It feels like if we’re talking about pip install django[expiremental] then we’re just creating a third party package which is no different than what we do today. I think that’s fine, but lacks ambition for what we should try to support.

I suspect, we’re trying to account for changes we can’t do in a third party package, but don’t want to commit to backwards compatibility. For that I think it’d be a mixture of having a dedicated package in Django like django.experimental or experimental modules in various modules, like django.contrib.auth.experimental. I prefer the package route rather than modules because I don’t expect this to be used much and it would allow us to see all the experimental stuff in a single area within the code.

I’m guessing at some point we would need a setting too. For example, if we wanted to change something about url resolving. So maybe we should have settings with the EXPERIMENTAL_ prefix?

[carltongibson]

I think there’s some appetite for an actual feature flags solution to be part of Django itself. This isn’t a minor addition so it’ll probably need the DEP process, but as an actual code area, it might be something you want to spend time on.

Whether we use such on bits of core itself is a separate topic. The Why here needs fleshing out. We already have a process: put it in a third party package, and then it possibly progresses to core. This system works: just as recent examples, template-partials, tasks, and fetch modes all went this route. Subatomic is doing similar.

Why do we need more? Not saying we don’t but I think the case for why is really not laid out.

Is the problem here social? Is it about messaging and communication? If so, we want to be careful that we’re not trying to implement the wrong kind (i.e. technical?) of solution. Is making the new-ish Ecosystem page on the site much better in fact the project here?

Stability is Django’s main selling point. (Reportedly, David Lord (Flask maintainer) was praising it just at PyCon US recently.) The danger is we break that. So how do we assure we don’t?

Of the routes suggested in the OP, I think the extras idea is the more feasible: we need experimental features (whatever that means) to not be tied to Django’s release process. (So is this just a way of blessing some packages that are in the pipeline? Again, is that really a comms issue?)

But extras or flags look like two different projects, not one.

(Just some initial thoughts. Excited to see where this goes!)

[Lily-Foote]

I largely agree with Carlton here. The one area where I wonder whether the third-party approach is suitable is beginner-focused features - for these we want the barrier to entry to be as small as possible. But maybe documentation can also address that?

[prafulgulani]

Thanks for feedback everyone.

@carltongibson

The Why here needs fleshing out. We already have a process: put it in a third party package, and then it possibly progresses to core.

I agree that third party packages work great for features that sit on top of Django like template partials or tasks. However, this project is meant for deep core (like work around url resolver) that are technically difficult to implement as a third party package.

@carltongibson

Is the problem here social? Is it about messaging and communication? If so, we want to be careful that we’re not trying to implement the wrong kind (i.e. technical?) of solution. Is making the new-ish Ecosystem page on the site much better in fact the project here?

Node.js(experimental flags) and Rust (feature gates) use this built-in approach specifically for these types of deep features. Improving the Ecosystem page is a great social goal for third-party discovery, but it doesn’t solve the technical need for a safe way to iterate on these internal core changes.

@carltongibson

But extras or flags look like two different projects, not one.

Sorry, I should’ve been clearer in my OP. Look at them as different strategies for isolation.

1. Logical Isolation (Direct Core): Experimental packages (like django.experimental or modules like django.contrib.auth.experimental as suggested by tim) and EXPERIMENTAL_ settings are included in the standard Django install. This maximizes discovery and testing but relies on the developer to respect the ‘experimental’ labels.
2. Physical Isolation (Package Extras): Everything in Path 1 is placed behind an installation barrier. This changes the pip command, which provides an extra layer of safety and explicitly signals to the user that they are entering an ‘Experimental Zone’ where stability isn’t guaranteed.

The goal is to decide if the extra safety of package extras should be added or not. Using Extras for isolation is a proven pattern in the Python ecosystem for maintaining lean, stable cores. Pydantic uses it to manage officially supported but optional features (pydantic[email]), and TensorFlow uses it to isolate complex, environment-specific logic (tensorflow[and-cuda]).

I want to build a framework that can support different implementations (namespaces, modules, flags or something else) we eventually decide on, but first we need to agree/disagree on the use of package extras.

@carltongibson

Stability is Django’s main selling point. (Reportedly, David Lord (Flask maintainer) was praising it just at PyCon US recently.) The danger is we break that. So how do we assure we don’t?

To ensure we don’t break that, the process would involve developing and testing these ideas as a third-party package first. By using these isolation techniques, we ensure that a Django user who doesn’t opt in for the experimental framework, never executes a single line of it.

@Lily-Foote

I largely agree with Carlton here. The one area where I wonder whether the third-party approach is suitable is beginner-focused features - for these we want the barrier to entry to be as small as possible. But maybe documentation can also address that?

That’s a fair point. While the extra install step is a small barrier, I believe the trade-off is worth it to ensure beginners don’t accidentally break their environments with experimental code. Great documentation is definitely a major focus of my project to bridge that gap. The first two points in the “Success Definition” Andrew @nanorepublica provided for this project specifically focus on updating and maintaining high-quality documentation, so rest assured he won’t let that slide!

[carltongibson]

Hi @prafulgulani,

I think we need to go a little more slowly to answer the Why question here.

@prafulgulani

… it doesn’t solve the technical need for a safe way to iterate on these internal core changes.

That’s exactly what needs to be shown.

@prafulgulani

… deep core (like work around url resolver) that are technically difficult to implement as a third party package.

This isn’t a good example. To swap in a custom resolver you’d just override resolve_request in a custom handler, and use that in your application. It’s perfectly doable in a package.

Almost without exaggeration, every one says, “Oh my feature needs to developed in core”. Once they realise that’s not happening, they then do the third party package. (I can immediately think of several cases of that happening just within recent cycles.)

It strikes me that the actual number of cases where a package isn’t actually viable are so rare as to not merit a whole “experimental features” framework. So, I come back to thinking, that’s not really why we might want one, or where the benefit really is.

That why is still sat there.

There’s potentially a major change to the way Django does things being suggested here. I think it’s important that get the rationale right. (Otherwise we’ll have little chance with the details.)

I hope that makes sense.

[rik]

When thinking about experimental, I’m thinking about landing code iteratively in Django.

I have not shipped or reviewed code in Django so my comments are based on my experience in other projects. The current process of landing things in one big pull request might be creating a toll on code submitters and code reviewers because each review cycle contains the whole shebang. By using an experimental namespace, some larger features could land in smaller batches that are more manageable for humans to write, review, update.

Only when all the pieces have landed can they be moved to a non experimental namespace. This part is creating extra work but it might be worth it if the rest of the work gets easier.

Such a process could also reduce stress when shipping major versions. A new feature with issues requires work to either fix them all in time for the release or to be backed out (which can be difficult to do cleanly). With an experimental namespace, none of that is needed.

This is the principle of Continuous Integration:

• Integrate code changes frequently to ensure that the integrated codebase is in a workable state.
• The longer development continues on a branch without merging, the greater the risk of integration conflicts.

People that participated in landing CSP and Tasks might have thoughts about this idea.

UPDATE: This process is independent of that code having started life as a third-party package.

[codingjoe]

Howdy!

I agree with Carlton here. Django is fairly good at inheritance and fairly stable even in its private APIs.

Off the top of my head I can’t think of any feature that couldn’t be built as a 3rd party.
I also fear that two different deprecation pipelines will confuse users.

As a maintainer, I am eagerly awaiting annual release cycles and a consistent clockwork in which to adapt to changes. This would actively work against that.

The only thing in the past that didn’t work without changes to Django core was ASGI. But we managed, and all subsequent async transformations are so stable that I haven’t noticed any problems with the deprecation cycle.

That being said, I wished for some packages to have broader adoption before merging them into core. Mainly the task framework, which isn’t bad, but also wasn’t very mature compared to its alternatives like Celery and even Dramatiq. It’s a bit painful to reach this maturity in Django, but this isn’t an issue with the deprecation process either.

Cheers
-Joe

[nanorepublica]

I am wondering if experimental flags ought to initially be limited to experimenting with changes in the existing core codebase. New features can rightly be trialed in a new package.

A well known example here is the auth.User model and it’s first/last name fields. The current general expectation for a User model is either to use the default one of have a custom one in your project. I know of 0 common third-party apps providing a custom user model to a project. While I could see a prototype to experiment with a new User model in a third-party package I wouldn’t expect much uptake due to the mentioned expectation.

However I could see the use of an experimental namespace to play around with what a new User model could look like inside Django, free of the need to maintain backwards compatibility for a period of time. This would give folks the opportunity to experiment with a new breaking API, from which we get to a known end state, then working out the migration steps becomes potentially easier? Yes this could be a third-party package, but when the API has already been established to be inside Django ought it not stay there in some manner?

This is borrowing from React Router’s idea of Future Flags: API Development Strategy | React Router

Personally I’m doing this with contrib.messagesas I think a refactor would improve the internals, so I am playing around with what the end design of the API looks like, then I can propose steps to get there. Again having an experimental namespace (or similar) would allow folks to opt in to trying this out.

[massover]

This isn’t a good example. To swap in a custom resolver you’d just override resolve_request in a custom handler, and use that in your application. It’s perfectly doable in a package.

I think “It’s perfectly doable in a package” is true as long as the expectation is that the 3rd party package may need to do some monkey patching. For example, if you dig into what changing resolve_request might look like if you were trying to add method routing, one thing you might see is the resolve method would need to change on URLResolver. URLResolver is called internally by get_resolver which doesn’t provide a way for the user to customize URLResolver. Perhaps this could be solved through “public interfaces” but I couldn’t see it.

I say this playing devil’s advocate though because I actually agree with your point. If I were to go back now and try and finish this pr for myself, I’d do it as a separate package first just monkey patching whatever is needed in django so I could just use it.

If we’re searching for rationale, maybe monkey patching = bad is something?

[carltongibson]

You’d just override BaseHander.resolve_request (src) and use your subclass in your wsgi.py/asgi.py file. It’s trivial. (The hard bit is the implementing it )

monkey patching = bad

I take it that subclassing isn’t monkey patching, so this thought doesn’t apply here. But we’re talking about implementing experiments. Even if you hold that monkey patching is bad, presumably for stable code, being able to swap out an implementation to show what’s possible is going to be a standard approach whether we have a flag solution available or not…

if flag: apply_monkey_patch()

One of the great things about Python is that code is just one more type of data.

[massover]

I take it that subclassing isn’t monkey patching, so this thought doesn’t apply here.

Absolutely agree: subclassing is not monkey patching, and django does provide a hook here that can help with some custom implementations here.

You’d just override BaseHander.resolve_request (src) and use your subclass in your wsgi.py/asgi.py file. It’s trivial. (The hard bit is the implementing it )

You can ignore me if you understood my original point. But if not, I am going to try and explain it again because you used the word “just”. A challenge is not about finding hooks like subclassing but about dealing with code that might not have a hook available for the user to override without patching. Yes, like you having pointed to, there is a method we can override to implement our own implementation of resolve_request. Other places in django call get_resolver. When my new HttpMethodHandler calls my new HttpMethodURLResolver, but the rest of django calls the framework’s URLResolver, something is not going to work as expected. Patching the URLResolver in django can be a start to implementing this feature, but I think there are other places in django that would need to be patched in order to implement http method routing as it is in my pr today (the real proof would be in a working third party package that I didn’t make, but I am trying offer my best perspective).

[carltongibson]

@massover Yes, you’re right — to do it fully you need to patch get_resolver. My assumption is that any resolver would need to be compatible with the existing one, and that it’s during request resolution that performance gains will be assessed.

But it’s somewhat moot. You think it’s a better example, I think it’s a worse one. OK, we can’t progress there.

The point is more about the claim (that’s always made) that we simply must develop something in core. It’s basically always false. You might say monkey patching is bad, but it’s just Python, and for experimental features it’s fine. We did some horrible things patching template loaders for template-partials, but it was fine. Folks could assess the proposal on an opt-in basis. We could fix the issues, unbound by Django’s release cycle, and then when it was proven, merge it.

I repeatedly hear desire for something in this space — to make the experimental features story better — but I’m sceptical that it’s a technical problem. (Third party packages are fine.) So I am concerned that we’ve not yet really identified what the goal should be, and that why I mentioned at the top.

[rik]

I’ve updated my post to clarify that this proposal to land code iteratively is independent of that code having started life as a third-party package.

[massover]

@carltongibson

But it’s somewhat moot. You think it’s a better example, I think it’s a worse one. OK, we can’t progress there.

The point is more about the claim (that’s always made) that we simply must develop something in core. It’s basically always false. You might say monkey patching is bad, but it’s just Python, and for experimental features it’s fine.

I actually intended to use my example to agree with you given the original discussion was around url resolver. I wanted to offer some insight as to why I (someone who doesn’t contribute to django) might have opted to make a pr and a dep (lol in retrospect), instead of just realizing “oh i can just patch django in a third party package to do what i need now”, and that’s the more realistic approach to getting it merged if that does make sense later anyway. I’ll stop over explaining now though, sorry for the distraction!

[prafulgulani]

@carltongibson

There’s potentially a major change to the way Django does things being suggested here.

I see this not as a change, but as an addition to existing methods, all current methods and procedures remain fully intact. The goal is to provide a structured, native pathway for features that would benefit from real-world usage and feedback during their design phase.

@rik

By using an experimental namespace, some larger features could land in smaller batches that are more manageable for humans to write, review, update.

I completely agree with @rik on this one. The current all or nothing approach creates a heavy toll on code submitters/reviewers. We can see this in Ticket-28805 (RegexpReplace database functions).

This ticket has been stalled for 9 years. The PostgreSQL and MySQL implementations were ready years ago. However, because Django currently requires simultaneous perfection across every supported database, the entire feature was on hold because of complex driver failures on Oracle.

A third-party package is not an option here, as Mariusz Felisiak noted in the PR review, proper ORM integration requires native as_{vendor} hooks inside core, rather than fragmented if/else vendor checks.

@rik

Only when all the pieces have landed can they be moved to a non experimental namespace. This part is creating extra work but it might be worth it if the rest of the work gets easier.

If we had an experimental namespace, this could have been landed in the smaller, manageable batches. The stable Postgres and MySQL pieces could have shipped years ago behind an experimental flag and the Oracle implementation could have been iteratively fixed over subsequent releases.

An experimental namespace provides a native pathway for features to mature through more usage and feedback. We see the need for this in Ticket-13240, where a patch was marked Ready for checkin but stalled because maintainers couldn’t agree on the API design. Alex Gaynor felt the proposed design would be confusing, but without a way to gather real-world feedback from users safely, the design debate reached a stalemate and was abandoned.

@nanorepublica raised a great point about established APIs, when a feature is already central to Django core, a third-party package isn’t a viable solution. We see this exact problem in Ticket-10227 regarding reverse OneToOne relationships. This has been open for 17 years because changing the reverse lookup behavior to return None instead of throwing a “Does Not Exist” exception would break backwards compatibility for legacy apps. An experimental namespace is the only way we can experiment with a new breaking API for established core systems natively, letting the design mature safely in place.

We see an even deeper structural need for this in Ticket-24312 (making it possible to import models safely at any time). Rewriting Django’s app-loading system to remove heavy reliance on import side effects is a massive architectural shift. A third-party package cannot solve this because it is bound by the very loading system it tries to fix. An experimental namespace provides exactly the playground @nanorepublica described to map out what the end design of such a breaking, core refactor looks like without breaking stable production environments.

@carltongibson

Is the problem here social?

it’s fair to ask if the problem here is social, better communication and ecosystem are definitely needed, but the “why” of this project is to solve cases where social consensus is impossible to reach without real word usage and feedback, if the problem was social then these tickets wouldn’t have been around this long!!

All in all, the more I research this space, the more I see how many use cases it can serve from safely navigating breaking API changes to building new features alongside the community. By gathering feedback and usage data from real-world projects, we invite more suggestions which eventually leads to more active contributors. This can possibly turn innovators and early adopters into active contributors and motivate others to switch to latest versions to experiment with new features.

[jacobtylerwalls]

I think the issue here is marketing. (I don’t see a convincing example where some feature must land in django/django–possibly under some kind of namespace–to be iterated on usefully.)

In rare cases, we’ve documented that a feature is private and not subject to the compatibility policy, e.g. TransactionTestCase.available_apps, but in that case it was a permanent private feature, not a temporary we reserve the right to break things for a couple versions, which is what I think we’re contemplating?

I’m most interested in thinking about the temporary extension points that we choose not to document simply because they are not stable, which is then self-defeating, because then only the hardcore source-spelunkers ever find out about them and provide feedback.

An example would be when I added a DEFAULT_FETCH_MODE extension point to Django 6.1. Surely it will take some other form than a module constant if an extension point is made public, so it’s temporary. But how will anyone find out about it if we’re forbidden from documenting it? Blog posts?

(Browsing the What’s New in 3.14? for Python, right up top is a mention of a feature with an experimental extension point.)

Maybe we need to be more explicit about the feedback we want to gather on major new features. We shipped preliminary support for composite primary keys while documenting what was lacking, including suggesting workarounds for expressing foreign keys to them. We shipped CSP middleware without adding support in the admin. We shipped tasks without a database worker. We shipped fetch modes without a global setting, and without copying fetch modes forward to related managers (the “to many” direction). We haven’t collected any of this information in one place to invite feedback, to my knowledge.

[prafulgulani]

@jacobtylerwalls

But how will anyone find out about it if we’re forbidden from documenting it?

You are completely right, if we aren’t documenting it, the majority of people will never discover them and hence no feedback regarding how to improve them.

@jacobtylerwalls

(Browsing the What’s New in 3.14? for Python, right up top is a mention of a feature with an experimental extension point.)

Python develops the unstable APIs along with the stable ones and they just mention a note for the unstable APIs that “this has no stability guarantees and may change or be removed.”

So, the question is, are there any problems with this approach?
Well, they are having a discussion on how documentation alone isn’t the right approach.
I want to quote a few things from that post because they highlight the issues faced when unstable APIs are documented without any isolation or runtime warnings. It’s a very thorough post that is highly relevant to what we are trying to solve, and I’d highly recommend it to anyone to go through it whenever you have some time.

The author points out that right now, there is no standard, machine-readable way to say an API is “published but not stable yet.” When maintainers want to add a new feature to a stable release, semantic versioning forces them into bad options: either hold the feature back completely, freeze an early design prematurely, or burn a major version bump later for a change that production users didn’t even care about.

He notes that documentation conventions like PEP 411 are a patchwork right now. They act purely as text-based rules, meaning they give no runtime warning, no runtime marker, and no signal that actually reaches the developer who installed the package, only the people who read the docs know about it.

So, he proposes that just like Python has a standard, type-checker-aware way to mark the end of an API’s life through @deprecated decorator (PEP 702), there should be an @experimental decorator for the start of an API’s life when something is published but explicitly unstable.

This will lead to a much cleaner workflow, it gives developers a clear runtime warning in their terminal instead of hiding it in documentation, it lets production teams explicitly block unstable code from going live, and it gives code editors a single standard marker to look for and flag automatically.

He further mentions Python libraries that are trying to solve this on their own. For example, scikit-learn has its own sklearn.exceptions.ExperimentalWarning + sklearn.experimental package, LangChain uses @beta decorator + LangChainBetaWarning. And libraries like NumPy, SciPy, and pandas rely on custom written text notes in their documentation or throw in generic, one-off warnings.

He also points out that several major languages like C#, Rust, Java and many more already ship built-in markers or flags to explicitly isolate unstable code.

Let’s also look at another angle, with the rise of AI assisted coding and agents being able to build entire applications, LLMs that are trained on the entire codebase and documentation know that the API is unstable but it’ll still use the unstable APIs anyway just to get the work done, regardless of the warnings in the documentation. The developer looks at the final outcome, and everything works fine and the code is pushed to production. A few weeks later the API is changed or removed, the application breaks.

And then that person will come on the forum asking for some sort of isolation and runtime warnings just like that developer did on the python forums (although he didn’t raise the issue because of LLMs) and I think this is one of the reasons why isolation is so important and I would even suggest going for package extras (pip install django[experimental]) so that there’s an installation barrier too. This installation barrier will force both the code assistant and the developer to acknowledge that they are in an experimental space.

Following this method, we wouldn’t have to change Django’s policy regarding undocumented features. Instead, by bringing unstable features under experimental space, we can safely document them and provide a direct link to a dedicated forum discussion where all the feedback can be gathered.

@jacobtylerwalls

I think the issue here is marketing.

Once these features are documented, they are self-marketing for anyone looking at the docs. However, to really drive awareness we shouldn’t limit ourselves to just blogs. We can use every means of communication Django has including newsletter, Discord, the forum, or podcasts, this way we can get a lot of feedback and usage before a feature is promoted.

Python gives us a lot of flexibility, and there are many paths we can take. It all comes down to finding the best way forward for the long-term, even if the implementation isn’t the most direct or natural approach at first. I would love to hear everyone’s thoughts on how we can best structure these while giving new ideas a safe place to grow.

I am open to any suggestions on the specific details to ensure the DEP 2 draft comprehensively covers how all of this will work. Specifically, it would be nice to get the details on what isolation methods should be used, how experimental phases will fit into the release cycle, and since this project comes as an alternative method, what is really needed for a feature to get accepted for this path?

Also, I wanted to know if there are any other reasons to keep features undocumented, or if protecting them from the deprecation policy is the only one?