[GSOC 2024] Auto-importing in the shell

Hi everyone!
I’m very entusiast that my proposal for GSOC’24 has been accepted! Thanks to @DevilsAutumn for feedback about a possible implementation of this feature and for helping me in writing the proposal.

This topic will be the one where I will update the progress of the PR: so every feedback is welcome. I’m very excited and entusiastic about this project and I’m looking forward to implement the auto-import for the shell.

Possible useful links:

In the coming days I will create a new PR that is going to be the one where the major part of the work will be done. In the mean time, if anyone wants to say something about this approach or the implementation described in the proposal, I’m here to listen all your feedback and I’ll be happy to review my work with your help.

Thanks,
Salvo Polizzi

4 Likes

Congrats @salvo-polizzi ! I’m really excited to work with you.
Just to be clear on what our final Shell will look like, I want to note down all the functionalities of shell_plus that we plan to add in Django Shell here. This way we can get to know the views of the community as well.

  1. By default, we’ll not print all model names in the Shell but it would be possible with a flag.
  2. Adding SHELL_PLUS_IMPORTS settings, see: shell_plus — django-extensions 3.2.3 documentation
  3. Adding SHELL_PLUS_PRINT_SQL settings, see: shell_plus — django-extensions 3.2.3 documentation
  4. Import all models using their full paths and then import the models in the order of the app names, starting with the first encountered app.
  5. User should be able to subclass the Shell command and override a method.
  6. By default, Django supports IPython, BPython, and standard shell runners. I believe we should enhance the current shell to allow users to add additional shell runners (jupyter notebook, ptpython etc) while preserving the auto-import functionality across all runners.
1 Like

Hi @salvo-polizzi , congratulations on being accepted. I proposed this project last year and I’m glad to see it is being picked up. I am not your mentor because I failed to log onto the GSoC panel, but I’m on hand to review your PRs.

Please review the past discussion with another potential candidate: Have Solved 80% of auto_importing Shell feature , and their PR: [gsoc] django_auto_import shell feature by Ammar-Munirr · Pull Request #17826 · django/django · GitHub .

In terms of features, I propose we don’t directly copy shell_plus. Instead, we should try to build a minimal, extensible design.

django-extensions’ design has “grown by committee” (or anarchy). Pretty much all PRs were merged, meaning its features are pretty haphazard. Its code shows that internally, with untested blocks, “secret” undocumented features, and general messiness. We’d rather make fewer solid features in Django, so we can maintain them for a long time.

Here’s how I’d refine Bhuvnesh’s suggestions:

  1. The shell command gains a new method, called, say, get_namespace(). This method returns a dictionary of names to objects, which are all added into the namespace.

  2. The default implementation of get_namespace() adds all models from apps.get_models(). We can have some debate about what to do with name collisions. But whatever we do, models in earlier apps need to take precedence, as that’s the precedence order used elsewhere, such as for management command lookup.

  3. The SHELL_PLUS_IMPORTS is not needed. Instead, we document how to extend get_namespace() to add other imports, something like:

# myapp/management/commands/shell.py
from django.core.management.commands import shell

class Command(shell.Command):
    def get_namespace(self):
        from django.core.urls import resolve, reverse
        
        return {
            **super().get_namespace(),
            "resolve": resolve,
            "reverse": reverse,
        }
  1. No feature to “display all imported objects”, nor an option to turn it on. I think we should stick with a simple message like “X objects automatically imported.”. Users can use globals() or shell-specific features to see all imported objects, if and when they care.

  2. Copying the functionality from SHELL_PLUS_PRINT_SQL is out of scope.

  3. This functionality should work in the supported shells as much as possible.

  4. There’s no need to add support for extending with other shells, as that already exists. One can override the command with a subclass and add an extra method with its name in the shells attribute. But this is undocumented, so it would be a nice bonus to make a side PR documenting how to use this feature.

Thanks again for working on this project, I’m excited to see how it goes!

3 Likes

Thanks @adamchainz for your valuable feedback, it clears also my mind about what to do. I also want to inform the community that I’ve created a draft PR where all the work will be done.

One other point @DevilsAutumn messaged me on Discord: we should make sure the default imports are loaded for -c / --command as well, which bypasses launching a shell interface by using exec():

The globals() argument can be expanded. That would allow quick one liners like:

$ ./manage.py shell -c 'print(User.objects.count())'
9001
2 Likes

Oh, yeah, we should also make sure the default imports are loaded for the stdin feature too, the block below the -c one.

1 Like


This is how the import of the objects is currently displayed, following the advice of @adamchainz. What do you think?