The work we have done so far has been great. I hope this new feature will be useful for most Django users. If you want to know more about it, see the related forum post, PR, and ticket.
Now, we are considering how to enhance the automatic imports. We have been discussing whether some default imports, such as django.conf.settings, could be useful for the end-users of the shell. We would appreciate feedback from the community so that we can agree on what should be automatically imported by default, avoiding a multitude of tickets requesting to add default imports.
One thing I keep thinking is missing from shell_plus at the moment is all the ORM bits and bobs. The main reason I use the shell is to test novel queries, so I often am a bit annoyed that I don’t have utilities and functions available, e.g. Subquery / OuterRef, Now / Case /When / Exists etc.
Would really love a star import of all these and more if it doesn’t have the chance to clobber too much stuff
Thank you for starting the discussion @salvo-polizzi , and good work so far on the automatic imports project.
In my experience, the shell is most often used for making or debugging database queries. So I think we should primarily consider imports to make that experience smoother.
For datetime, I propose we do:
from datetime import *
import datetime as dt
from django.utils import timezone
datetime.timezone will be shadowed by django.utils.timezone, hence the dt alias for the datetime module as well. This is a practice I’ve recommended and seen adopted by many: How I Import Python’s datetime Module - Adam Johnson .
For ORM tools, I propose we do @tom’s suggestion:
from django.db.models import *
from django.db.models.functions import *
I don’t think there’s much risk of clobbering/shadowing, because we’re starting with an empty namespace, and we’ll put these default imports before models and anything else users might add.
I’m also +1 to from django.conf import settings. Whilst it’s not ORM-related, I’ve realized I use it fairly often.
I checked through my IPython history for any other key imports. The only one I spotted was from decimal import Decimal I think most projects use DecimalFields, so being able to define values without an import is going to be valuable.
Do we think that we need to document everything that is automatically imported or say something along the lines of “the shell has a series of automatic imports to assist making queries, you can see the full list by calling globals()”?
I personally like that currently shell_plus lists all the imports it has automatically performed (and in what order), it means the context for what you are working with is right there on the screen with you with out having to call an extra function or remember which function to call to see the list (I don’t use globals() regularly, if I ever have).
I also mention the order, because there have definitely been a time or two where a project has 2 models name the same thing (from migrating to wagtail if I remember correctly) so listing the order helps to know which, if any, models are currently imported.
I am against printing all the imports by default, since on even modest projects, they can be hundreds of lines. That’s why I asked @salvo-polizzi to only implement the “N objects imported automatically” message the PR currently has. Perhaps we could enable it with extra verbosity (-v 2) though? It would need some care too since user code can add objects to the namespace that don’t have a sensible import path to display.
I think a small docs section is worth it. Perhaps a sentence about the database queries rationale and a list of the effective import statements being run. globals() will list everything, so it won’t be as compact and understandable as from django.db.models import *.
I like the idea of autoimporting extra modules. However, I suggest narrowing the datetime imports, as its usage may not be considered widespread or common in every shell interaction. Specifically, I believe the following would be sufficient:
from datetime import date, datetime, timedelta
I would recommend omitting import datetime as dt. While I acknowledge Adam’s influence and recognize that it may be adopted by many, it still seems somewhat niche to me. I would suggest leaving this option to the user for customization.
Then I would like to suggest that we autoimport get_user_model and define User = get_user_model(), assuming that django.contrib.auth is listed in INSTALLED_APPS. In my experience, the second most common usage of the shell is to make an existing user staff or superuser.
One more suggestion I thought of: from django.db import transaction. I had a need the other day to use transaction.atomic() whilst live-coding with the ORM.
@adamchainz What would be a common use case for transaction in the shell?
In my (biased!) experience, the shell is used to inspect queries and see how objects and method behave, but I have not (never!) “debugged” transactions in the shell (I have done so via test cases where I can manually force the interleaving I need). But maybe I’m missing use cases? I would love to know more!
Therefore, I suggest not including from django.db import transaction by default. I believe that most large projects can function effectively without manually defining or managing transactions.
Additionally, I often find myself needing from django.db import connection, reset_queries, but I am hesitant to push for these to be imported by default, as this may enter the realm of personal preferences and habits.
Personally, I don’t mind either way on django.db.transaction. I do however want to +1 the django.db.connection and reset_queries, these are quite useful when debugging queries.
I’m going to be controversial and suggest dumping all models for all installed apps into the shell, with django* first (and then thus being overridden by later stuff). Maybe just in the order from INSTALLED_APPS (because it’s easy to explain), and with a defined file it loads from automatically, even better if that file is generated by startproject.
Plus all the stuff Adam and Tom said above with datetime and Q/F, etc. I would also like to propose importing Path, json, and (if available) tqdm. I would like numpy as np but I’ve seen that just trying to import numpy is quite expensive so I would recommend against that.
Since we have the ability to allow folks to customize this to their own needs, it feels prudent to be very strict on what goes in by default.
Overtime, if it’s to narrow for folks, people will copy/paste a custom shell management file to add to the import namespace. When that happens, we’ll have a more concrete example of what people are using and for us to discuss what else should go in.
I have to imagine removing things from this would be difficult. Specifically because of the -c argument, for example: shell -c "print(settings.SOME_EXAMPLE)". Arguably, removing things that are imported would be backwards incompatible and would take a few releases to remove.
I would say the reverse. If you can customize it, you can be quite liberal in what you throw in there, as the user can override it to shadow what is added by default anyway.
I’m confused… I feel like we’re talking about different things here. The shell is for interactive use, why would this be included in any backwards compatibility discussion?
