Implement default handling of --verbosity in management commands

Django management commands expose a verbosity flag (-v, --verbosity with values {0,1,2,3}) by default (implemented in the BaseCommand class), but this flag has no effect unless the developer of the command has cared to do so.

E.g. When using default Python logging in management commands (or third-party libraries used in a command), logging adheres to the settings in Django’s settings.LOGGING dict. To me it seems intuitive that a management command called with -v 3 will switch (for the output of that command) the log level to logging.DEBUG, but it doesn’t.

I would like to propose a default implementation in Django’s BaseCommand, aiding developers to handle verbosity control with minimal effort.

My observations:

  • Django exposes a default interface for verbosity control in management commands, implying to the end user that this does something useful. The fact that this flag is always available raises end-user expectation.
  • The implementation of verbosity is left to the developer, but no guidelines or hints are given. The documentation doesn’t spend a single word on logging or verbosity.
  • The opaque descriptions of the verbosity levels in the management command flag (Verbosity level; 0=minimal output, 1=normal output, 2=verbose output, 3=very verbose output) don’t map naturally to existing standards like Python’s builtin logging log levels. This raises the bar for developers to do something useful with the flag.
  • Willing developers of management commands in third-party apps might implement verbosity control, but as no guideline, preferred approach or basic implementation is available, everyone has to re-invent the wheel somehow. This again raises the bar for developers to actually do this. All these implementations will vary wildly in their interpretation of how verbosity control is applied.
  • Less willing (or ignorant) developers won’t implement any verbosity control at all. This happens a lot. The lack of guidance might be a reason for this.

As an end-user of (third-party) commands, the lack of verbosity control despite a CLI flag being available has annoyed me to no end.

As a willing developer of management commands, I find that I either need to look back to previous work for easy handling of the verbosity flag, or just think up something new each time. Over the years I might have created a dozen slightly different ways to handle this.

Thus I propose to create a default implementation for handling verbosity. My basic idea would be to implement in Django something like:

  • Map --verbosity values to Python logging levels (e.g. 0 → logging.ERROR, 3 → logging.DEBUG).
  • The BaseCommand class should automatically create a logging logger that is easily usable from methods in the command, e.g. self.log.info(“foo”). This logger should adhere to settings.LOGGING, but enforce the log level set in the –-verbosity flag. It could possibly also enforce output to stdout (without doubling up with existing configuration that sends to stdout?).
  • The application of the verbosity level to loggers could be configurable in code: only the logger local to the management command file, to the django app the command belongs to, or even the root logger.
  • Promote use of the self.log object (or the logging module) in Django documentation about custom management commands.
  • Maybe document (or even implement) a preferred way to combine the verbosity level with Command.stdout.write().

Would this be any good? Other ideas and suggestions welcome of course.

2 Likes

This sounds like a good idea to me.

Great points @whyscream.

I wonder if a good starting point would be by offering a mixin, like VerbosityLoggingMixin with good documentation and that would offer the benefits you suggested? Alignment with log levels, handy access to self.log. OTOH it would be opt-in and not affect existing commands.

I started a proof of concept implementation in github:whyscream/django-logging-management-command and will try to work on it.

@rodbv Totally agreeing that a separate package will indeed not have any use, as it is opt-in only.

We have an accepted ticket to replace self.stdout/self.stderr in the commands with logging, with some status updates there and a WIP branch.

I suggest harmonizing your ideas for verbosity with that rewrite.

1 Like

Thanks for the reference @jacobtylerwalls . After reading the issue and some unfinished previous efforts, I’m unsure if that ticket is the way to go. Particularly the integration of self.{stdout,stderr}.write() with the logging framework seems strange to me, as the output of those commands is clean: no logging ‘window dressing’ like timestamp, log level, etc. This makes self.stdout.write() ideal when your command generates pre-formatted output (from tabulate · PyPI, or json.dumps(..., indent=2) to name a few), which is typically stuff that 1) you don’t want in log files, and 2) doesn’t look/behave well with additional output on the line when sent to stdout.

I would keep these methods as-is, but add an optional hook to let the output work with the verbosity flag: self.stdout.write("my output line", ending="\n", verbosity=1) would output the line to stdout only when verbosity is set to >=1. Leaving verbosity at None (the default), would output unconditionally, as it is now.