CLIs: an Underrated Interface, and Why You Should Use Click

An Underrated Interface

So now that we have a system with an API — a JSON web API, to be precise — we need to put it in front of human eyes. JSON is great for programs, but pretty lousy for humans — you can't just sit down and browse a long, unformatted document and figure out what it describes. There are two ways to go about it: we can develop a command-line interface, or a graphical one. We'll do the latter in the next chapter; first, though, we'll cover the former — not only because it's much easier to implement, but because I think it doesn't get the credit it deserves.

On one hand, a CLI is much simpler to develop than a proper, graphical user interface: the latter requires knowledge of the underlying windowing system, rendering libraries, etc. — whereas the former is basically a text-based, conversational version of our API. On the other hand, it's much simpler for users to automate: writing a script that runs a few commands is trivial, while automating a mouse to click buttons in a certain order is possible, but clunky.

I think there's a common misconception in software development, according to which the user is stupid. In the C++ world, Scott Meyers has what he calls the most important guideline in software design: "make interfaces that are easy to use correctly, and hard to use incorrectly." The first part is pretty obvious — but the second is more subtle: it means that if the user reads some documentation, genuinely tries to make something happen, and fails — the fault is with the interface, not with the user. Nobody comes to work thinking "today I'll do a particularly bad job"; so if we design our interface to be used by people who are willing to put in some effort, we can actually assume they know how to work a terminal.

I'll give you an example from my army days. The development department would get a request from the operations department, and proceed to develop an entire system, with all sorts of nice buttons and controls — basically treating the operations people as monkeys who would push the buttons, while all the magic would happen behind the scenes. At some point, I was working on such a system — but I actually made it available as a CLI, and even as a Python library; so I showed it to them (one of them, who had a math degree, was actually sharper than most of us) — and lo and behold, they learned it, and started using it much more extensively, because they could close the loop on their own. In fact, they started running all sorts of experiments that would never warrant half a year of development, but were easily worth a weekend of playing around with different parameters.

This is what's called the gulf of execution: the distance between what you want to do, and how you actually get to do it — and a CLI dramatically shortens this gulf. It's also a handle: if you're familiar with Apple, you might know that they put handles on their desktop computers, which were supposed to sit on tables and never move. The reason, according to Jony Ive (talking about the iMac), was that people were intimidated by computers, and having a handle implied there was at least one thing you could do with this strange object — so it wasn't as scary. Similarly, when people start working on a project, the first thing they do is read the README file, and the second thing they do is run the CLI: it gives them a quick feeling of accomplishment ("OK, I made it work! All is right with the world!") before they set out to understand what exactly is going on.

As an aside, on Unix, a lot of shared object files (.so, the equivalent of DLLs) can actually be executed, and most of them will print out their version — which is something that blew my mind the first time I saw it, because it's much nicer than having to open the binary and figure out which string is the version string. Almost every project I do has a CLI, if only to query its version or run its tests.

In fact, Python itself is full of CLIs: you can run a module as a script by using the -m switch, and many standard modules come with such a script — like json.tool, which pretty-prints JSON, or calendar:

$ echo '{"name": "Alice", "languages": ["python", "bash"]}' | python -m json.tool
{
    "name": "Alice",
    "languages": [
        "python",
        "bash"
    ]
}
$ python -m calendar 2026 10
    October 2026
Mo Tu We Th Fr Sa Su
          1  2  3  4
 5  6  7  8  9 10 11
12 13 14 15 16 17 18
19 20 21 22 23 24 25
26 27 28 29 30 31

And notice how json.tool actually read its input from a pipe; it's a tiny program that does one thing, and is composable with whatever produces the JSON — whether it's a file, curl, or another program entirely. This composability is at the very heart of the matter.

So here's the plan: we'll start with the anatomy of a command — the arguments, the standard streams and the exit code, and the Unix philosophy of pipes; then we'll do it the hard way, parsing sys.argv manually, and then with argparse, before implementing our own decorator-based framework that infers what to do from function signatures, in a callback to our chapters about functions and decorators. Then, we'll cover Click, which does all of this (and more) beautifully; and finally, we'll see how to turn our script into a proper, full-fledged command.

Anatomy of a Command

I like to think of a graphical user interface (or GUI, for short) and a command-line interface (or CLI) as two different interfaces to the same thing. Your desktop, for example, is just a directory (~/Desktop) whose contents are "spread out" on your initial screen; and both the file manager and the terminal app work with the same operating system, accessing the same files. They just provide different interfaces to it, with the CLI using a keyboard to type out commands instead of a mouse to click on windows. And what's more, those commands are so much more flexible and automatable, that they're worth learning about.

In fact, let me tell you what a command is: it's a program — like ls or cat — followed by a list of arguments that further configure its behavior. When it's running, it can read from the standard input (or stdin), and write to two output channels: the standard output (or stdout) and the standard error (or stderr). When it's done, it returns an exit code: 0 indicates that everything went well, and any other number indicates that an error occurred, with different values indicating different errors, which can be interpreted differently by different programs.

Why two output channels, you ask? It's a good question: the reason is that they let us distinguish between the "regular" output of a program, and the metadata about the program's execution — like errors, warnings, or progress reports. This way, we can silence either one of them, or redirect them separately — sometimes I want to see the errors, and don't care about the output, and sometimes I want the opposite.

Now, in C, those arguments are passed to main as argc and argv; and in Python, they are made available through sys.argv, a list of strings with the first item being the name of the script:

$ cat args.py
# args.py
import sys

print(sys.argv)
$ python args.py
['args.py']
$ python args.py hello world
['args.py', 'hello', 'world']
$ python args.py "hello world" --name=alice *.py
['args.py', 'hello world', '--name=alice', 'args.py', 'divide.py']

Notice how the shell is the one doing the parsing, before Python is even invoked: grouping two words into a single argument with quotes, expanding *.py into all the matching files. Also, notice that all arguments are strings; there is no such thing as a number on the command-line, so any conversion is something that we have to do ourselves.

Another fun convention, at least in Unix, is that if no script is specified, python opens up an interactive session — and similarly, if no file argument is specified for many programs, like cat or sort, they default to reading from the standard input. In other words, if you pass in some arguments, I'll work on them; but if not, let's work together interactively. Anyway, let's implement divide.py:

# divide.py
import sys

x, y = sys.argv[1:]
try:
    print(int(x) / int(y))
except ZeroDivisionError:
    print('ERROR: cannot divide by zero', file=sys.stderr)
    sys.exit(1)

Here, print(..., file=sys.stderr) prints something to the standard error, and sys.exit(n) ends the program with the exit code n. We can test it out:

$ python divide.py 1 2; echo "exit code: $?"
0.5
exit code: 0
$ python divide.py 1 0; echo "exit code: $?"
ERROR: cannot divide by zero
exit code: 1
$ python divide.py 1 0 2> /dev/null; echo "exit code: $?"
exit code: 1
$ python divide.py 1 2 > result.txt; cat result.txt
0.5
$ python divide.py 1 2> /dev/null; echo "exit code: $?"
exit code: 1

Going through it line by line, we see that $? is a special variable, indicating the exit code of the previous command; 2> /dev/null is a way to silence errors only (so the exit code still indicates that the command failed); and > result.txt is a way to capture only the output, so any errors will still be printed to the screen. And if there's an uncaught exception, Python will actually print a traceback to the standard error and exit with code 1 — which is exactly what happens when an argument is missing and the unpacking fails.

Here's a fun exercise: cat is a program that concatenates files and prints them out, but if you pass in - as the file name, it switches to reading from the standard input instead. So cat a - b will print a, then wait for you to type something, echo it back at you (until you hit Ctrl+D, which indicates end of file), and then print b. What do you think cat a - < b will do?

$ cat a
a
$ cat b
b
$ cat a b
a
b
$ cat a - < b
a
b
$ echo "I'm in the middle!" | cat a - b
a
I'm in the middle!
b

The answer is a and then b, because < b is a way to turn the file b into cat's standard input, which is exactly what the - argument is waiting for. Similarly, if you pipe a line into cat a - b, it will end up right in the middle. Now, none of this is very useful, but I hope it demonstrates just how nuanced and powerful this concept of a command can be. We've seen that it's made up of arguments, stdin and stdout, but we've also seen that they're all just files, really, and it's the shell that decides which file goes where.

Pipes and the Unix Philosophy

That's why the Unix software design is such a marvel. It's an ecosystem — and in it, everything is a file: open, read, write and close are the four basic functions that work on files, but also on directories, special files like /dev/null and /dev/urandom, and even devices. Every program does one thing, and does it well; and a simple but powerful abstraction called a pipe, or |, connects one program's stdout to the next program's stdin.

sort might seem stupid: why would you want to run it, type some lines and have them sorted? But in a pipeline, as a way to sort the output of some other program, it suddenly makes sense. The same is true for uniq (which collapses adjacent duplicate lines), wc -l (which counts lines), head (which keeps the first lines) or tr (which translates characters).

There's a great story about this. In 1986, Jon Bentley's Programming Pearls column asked Donald Knuth to demonstrate literate programming by solving a problem: given a text, print the n most frequently used words in it, along with their frequencies. Knuth wrote a careful, beautiful program — several pages long, mind you — that solved the problem with a clever, custom data structure. When Doug McIlroy was asked to review it, he sent back a six-stage shell pipeline. It's not a dig at Knuth, far from it — just an amusing illustration of the power of Unix. Here it is, counting the words of the Zen of Python:

$ python -m this | head -3
The Zen of Python, by Tim Peters

Beautiful is better than ugly.
$ python -m this | tr -cs A-Za-z '\n' | tr A-Z a-z | sort | uniq -c | sort -rn | head -5
  10 is
   8 than
   8 better
   6 the
   5 to

Let's break it down. python -m this simply prints the Zen of Python, so we can play with it. tr -cs A-Za-z '\n' replaces every character that isn't a letter (-c, for complement) with a newline, and squeezes multiple newlines into one (-s); so every word ends up on its own line. Then, tr A-Z a-z lowercases everything, and sort brings all the identical words together — which is necessary, because uniq only collapses adjacent duplicates. uniq -c does just that, but also counts how many of each there were; and sort -rn sorts it numerically (otherwise 10 would come before 8) and in reverse, so the most frequent words are first. Finally, head -5 takes the top five — actually, McIlroy used sed ${1}q, which does the same thing, but accepts the desired number as an argument to the script.

Now, imagine what it must be like to have this fluency: to sit down at your desk and have a task, and simply translate your thoughts into what you want to happen. This is a big part of the difference between people who can solve problems, and people who can merely write software (and do so slowly).

Here's the same thing in Python, and as a proper filter:

# wordfreq.py
import collections
import re
import sys

n = int(sys.argv[1]) if len(sys.argv) > 1 else 10
words = re.findall('[a-z]+', sys.stdin.read().lower())
for word, count in collections.Counter(words).most_common(n):
    print(f'{count:7} {word}')
$ python -m this | python wordfreq.py 5
     10 is
      8 better
      8 than
      6 the
      5 to
$ python -m this | python wordfreq.py 100 | grep -c .
87
$ python -m this | python wordfreq.py 100 | sort -k2 | head -3
      2 a
      3 although
      1 ambiguity

It takes n as an argument, reads from stdin, writes to stdout, and uses collections.Counter, which is similar to sort | uniq -c, and its most_common method, which is similar to sort -rn | head.

And it plugs right into the Unix ecosystem, on both ends: we can feed it the Zen of Python by using python -m this, count how many non-empty lines are in its output with grep -c . (there are 87 distinct words), and even re-sort its output by the second column with sort -k2. The only difference might be in the order of words with the same frequency — most_common preserves the order in which they were first seen, whereas sort -rn falls back to comparing the whole line (in reverse, because of -r).

The point of this exercise is not to convert you to a 70s mindset; it's to drive home the importance of software design, and illustrate how a small, seemingly insignificant change can make a program a part of an ecosystem, and thus increase its potential impact tremendously. If it reads from stdin, writes plain lines to stdout, keeps its errors on stderr and returns a meaningful exit code — then it becomes a citizen of this amazing world, and can be a building block in things you've never even conceived of.

The Hard Way

To start, let's consider the most basic example of a command-line interface: a simple calculator, supporting two commands, add and mul, each receiving two integers as arguments. Our implementation is going to be very barebones, but it's going to serve well enough for our purposes:

# calc.py
import sys


def usage(argv):
    print(f'USAGE: {argv[0]} (add | mul) X Y', file=sys.stderr)
    return 1


def main(argv):
    if len(argv) != 4:
        return usage(argv)
    command, x, y = argv[1:]
    try:
        x, y = int(x), int(y)
    except ValueError:
        return usage(argv)
    if command == 'add':
        print(x + y)
    elif command == 'mul':
        print(x * y)
    else:
        return usage(argv)
    return 0


if __name__ == '__main__':
    sys.exit(main(sys.argv))

Let's go over it line by line. We start with a usage function, which is used to print the instructions on how to use our program — and that's pretty much all the boilerplate we'll have in this example. Then, our main function expects an argv parameter, which we'll pass in from the entry point, if __name__ == '__main__'; this is where we'll receive the command and its arguments. First, we check that exactly three arguments were passed in: the command itself, and two numbers to operate on. We then convert these two numbers to actual integers, since everything arrives as a string; and finally, we use a cascade of if and elif to dispatch our command — either add or mul, or neither. In the last case, or if anything goes wrong, we simply print our usage instructions to the standard error stream, and return a non-zero exit code to indicate that the program failed. If it succeeds, we return 0. Notice that in our main entry point, we don't do anything except passing sys.argv in and translating main's return value into an exit code with sys.exit. And it works:

$ python calc.py add 1 2
3
$ python calc.py mul 3 4
12
$ python calc.py pow 2 10; echo "exit code: $?"
USAGE: calc.py (add | mul) X Y
exit code: 1
$ python calc.py add 1; echo "exit code: $?"
USAGE: calc.py (add | mul) X Y
exit code: 1

Now, there's a small but interesting pattern here that I'd like to point out. A lot of people would write all of this code under the if __name__ == '__main__', or at least, have the main function read sys.argv directly. The first approach makes it impossible to call the main function from tests, or anywhere else in our code (I suppose you could import the module as __main__, but please don't); and the second one makes it dependent on a global variable, so to test it we'd have to replace it with some value and remember to restore it afterwards. By passing argv as a parameter, we achieve a nice decoupling: a small example of what's known as dependency injection, in that our function receives what it needs, rather than reaching out to get it. As an aside, it also lets us play with it in the interpreter:

>>> import calc
>>> calc.main(['calc.py', 'add', '1', '2'])
3
0
>>> calc.main(['calc.py', 'add', '1'])
USAGE: calc.py (add | mul) X Y
1

And from a test, it's just a function call, plain and simple: its return value is the exit code, and it prints the usage instructions to the standard error stream if necessary. But why do we need a command-line interface for internal code to begin with? If we're working on a project, we might need to run its tests, trigger some specific functionality, or debug our API; and if we're onboarding someone new to the project, we might want to give them a simple way to get started. However, all of these cases can become pretty cumbersome, pretty fast: if we have many commands, we'd have to support a pretty long elif-chain; and if these commands expect different arguments, we'd have to parse them accordingly — for example, "if the command is foo, read three more arguments; and if it's bar, read two more", in addition to supporting any options, defaults and help messages that might be necessary. Needless to say, nobody enjoys implementing such logic — so Python provides us with a way to do it with argparse.

The idea is pretty simple: we create an ArgumentParser, use its methods to describe what arguments we expect, and then invoke it on our arguments to parse them, and pass them on to our business logic — manually, of course. To demonstrate this, let's rewrite our calculator to sum or multiply integers, and use a -m flag to specify which behavior is desired:

# sumul.py
import argparse
import functools
import operator

parser = argparse.ArgumentParser(description='Sum (or multiply) two integers or more.')
parser.add_argument('x', type=int, help='the first argument')
parser.add_argument('ys', type=int, metavar='y', nargs='+', help='the other argument(s)')
parser.add_argument('-m', '--multiply', action='store_true', help='multiply instead of add')
args = parser.parse_args()

numbers = [args.x, *args.ys]
if args.multiply:
    print(functools.reduce(operator.mul, numbers))
else:
    print(sum(numbers))

And it works:

$ python sumul.py -h
usage: sumul.py [-h] [-m] x y [y ...]

Sum (or multiply) two integers or more.

positional arguments:
  x               the first argument
  y               the other argument(s)

options:
  -h, --help      show this help message and exit
  -m, --multiply  multiply instead of add
$ python sumul.py 1 2 3
6
$ python sumul.py -m 1 2 3 4 5
120
$ python sumul.py 1 two; echo "exit code: $?"
usage: sumul.py [-h] [-m] x y [y ...]
sumul.py: error: argument y: invalid int value: 'two'
exit code: 2

So argparse is able to convert types for us, enforce that y can appear one or more times, and even generate a nice help message for us. If we provide invalid input, it even prints the usage and exits with the standard exit code of 2, used to indicate incorrect usage; and generally, it helps. It's just a very bare-bones, low-level way of doing things: if you look at the interface, what is this action='store_true' about? What's a metavar? What does nargs='+' mean? And this is just the beginning; we still have to unpack everything and dispatch it ourselves — and if we want subcommands, that means subparsers, which are more of the same. No wonder that when our tools are so crude, it becomes so frustrating that people just don't bother making CLIs at all.

So why is it so popular? Well, if you search the internet on "how do I write a command-line interface in Python", argparse is the first thing that pops up — so that's what people use. But just because the majority of people do something, it doesn't mean it's the right thing to do — and there are enough people in the majority as it is. My position on the matter is quite simple: there is absolutely no reason to use argparse, when we have Click — that is, unless you have a very strict constraint of not using 3rd party libraries, in which case I guess you're out of luck. If you ever find yourself in a situation where you think "well, I have to use argparse here, because…" — please, email me, and let me know what exactly is it that you have to use it for, because in my opinion, you don't; and if I'm wrong, I'd like to know.

But before we get there, I want to take a different approach: let's develop something better ourselves.

A Framework of Our Own

Daydream for a moment, then: what would we like to write? Just functions, maybe — and somehow indicate, with a decoration or something, that they're to be considered as commands? Then, maybe we could infer their argument structure from their signature: anything with a default argument can be considered optional, so it maps to a --name=value option (or, if the default is a boolean, a flag), while anything without one is required, so it's a positional argument; and maybe we can even add annotations to specify what type to convert them to?

You know, like in the Metaclasses chapter, where we registered command classes (first with a decorator, then a metaclass, then __init_subclass__), and said "never mind how the arguments are parsed" — well, now we do mind. And here, functions are much lighter than classes, since their signature already describes an interface, which is exactly the point I was trying to make in the first Functions chapter. So:

# cli.py
import inspect
import sys


class CommandLineInterface:

    def __init__(self):
        self.commands = {}

    def command(self, f):
        self.commands[f.__name__] = f
        return f

    def main(self, argv=None):
        argv = sys.argv if argv is None else argv
        if len(argv) < 2 or argv[1] not in self.commands:
            return self.usage(argv)
        f = self.commands[argv[1]]
        try:
            args, kwargs = self.parse(f, argv[2:])
        except (TypeError, ValueError) as error:
            print(f'ERROR: {error}', file=sys.stderr)
            return self.usage(argv)
        f(*args, **kwargs)
        return 0

That is, a CommandLineInterface class with a command decorator and a main method. The latter expects an argv list (defaulting to sys.argv for the same testability reasons as before), looks up the requested command, parses the rest, calls the function, and returns 0 if it succeeds. If there's any problem with the parsing, it prints an error and the usage, and returns 1.

The command decorator is the simplest possible; it simply records the function under its __name__, and returns it as is, so it can be called. So far, it's pretty similar to what we had before; the interesting part is parse, where we actually bind the arguments:

# cli.py (continued)
    def parse(self, f, words):
        signature = inspect.signature(f)
        args, kwargs = [], {}
        for word in words:
            if word.startswith('--'):
                key, equals, value = word[2:].partition('=')
                kwargs[key.replace('-', '_')] = value if equals else True
            else:
                args.append(word)
        bound = signature.bind(*args, **kwargs)
        for name, value in bound.arguments.items():
            annotation = signature.parameters[name].annotation
            if annotation not in (inspect.Parameter.empty, bool):
                bound.arguments[name] = annotation(value)
        return bound.args, bound.kwargs

Let's go over it: we iterate over the words that follow the command name. If a word starts with --, then we peel it off and consider it a keyword argument (replacing any dashes with underscores, and assuming any bare --flag is supposed to be True); anything else is considered a positional argument. We then use signature.bind to do the heavy lifting: given some arguments, it attempts to match them to the function's parameters in the same way a call would, raising a TypeError if some argument is missing or doesn't match, and providing error messages like "missing a required argument: 'y'". Then, for every bound argument whose parameter has an annotation (other than bool), we convert it by invoking that annotation: so int('1') is 1, but int('two') raises ValueError. That's annotations used at runtime, by the way; it's not what most people use them for, but like the validator we wrote in the Type Hinting chapter, it's perfectly legal. As an aside, the original version of this exercise actually used inspect.getfullargspec, and expected key=value arguments; inspect.signature is the modern interface, and it also follows __wrapped__, so any commands decorated with functools.wraps would still have their original signature exposed (remember that from the Decorators chapter?).

Now, let's see how we generate the usage:

# cli.py (continued)
    def usage(self, argv):
        print(f'USAGE: {argv[0]} COMMAND [ARGS]', file=sys.stderr)
        for name, f in self.commands.items():
            params = []
            for param in inspect.signature(f).parameters.values():
                if param.default is inspect.Parameter.empty:
                    params.append(param.name.upper())
                elif isinstance(param.default, bool):
                    params.append(f'[--{param.name}]')
                else:
                    params.append(f'[--{param.name}={param.default}]')
            print(f'    {name} {" ".join(params)}', file=sys.stderr)
        return 1

We iterate over each signature's parameters: no default means it's a required argument, so we use an uppercase placeholder like X; a boolean default means it's an optional flag, so we use [--name]; and any other default means it's an optional argument with a default value, so we use [--name=default].

And that's pretty much it; now, we can do:

# example.py
import sys

from cli import CommandLineInterface

cli = CommandLineInterface()


@cli.command
def inc(x: int):
    print(x + 1)


@cli.command
def add(x: int, y: int):
    print(x + y)


@cli.command
def hello(name='stranger', uppercase=False):
    message = f'Hello, {name}!'
    print(message.upper() if uppercase else message)


if __name__ == '__main__':
    sys.exit(cli.main())

Notice how example.py doesn't even need to know about argv: it's just annotated functions with a decorator on top, followed by sys.exit(cli.main()). We can then do:

$ python example.py; echo "exit code: $?"
USAGE: example.py COMMAND [ARGS]
    inc X
    add X Y
    hello [--name=stranger] [--uppercase]
exit code: 1
$ python example.py inc 1
2
$ python example.py add 1 2
3
$ python example.py hello
Hello, stranger!
$ python example.py hello --name=Alice --uppercase
HELLO, ALICE!
$ python example.py add 1
ERROR: missing a required argument: 'y'
USAGE: example.py COMMAND [ARGS]
    inc X
    add X Y
    hello [--name=stranger] [--uppercase]
$ python example.py add 1 two
ERROR: invalid literal for int() with base 10: 'two'
USAGE: example.py COMMAND [ARGS]
    inc X
    add X Y
    hello [--name=stranger] [--uppercase]

And notice how the usage is generated, not written; and how hello's default values are exposed as options, just like we wanted. And if we make a mistake, like providing a non-integer y or omitting it entirely, we get the correct error message from bind or int, followed by the usage again.

Of course, there's a lot of room for improvement: we don't get help texts from docstrings, subcommands, prompts, environment variables, friendlier errors or shell completion; but each is just an afternoon's work. What's important about this exercise is the idea: functions as the unit of interface, decorators for registration, and introspection for inference. There's a library called Click, which does all of the above (and much more), and then there's Google's Fire, which takes this idea to its logical conclusion. Give it a module, and it will generate a full-fledged CLI for it, with no decorators at all, inferring the commands, arguments and options from the functions themselves. Whenever you encounter a library that makes you think "wait, how is this even possible?", remember that it's open source, and go read its code.

Click

At this point, I'd like to get a bit meta: I don't usually spend a lot of time teaching libraries, because it's a bit like reading a tutorial out loud. But in this case, it's worth it — Click is a masterpiece of API design. It's written by Armin Ronacher, who also wrote Flask, Jinja, Werkzeug and other cool projects (collectively known as Pallets); and once you see a person do something amazing, they tend to create ten more, so it's usually a good idea to check out everything else they did. It might be a bit underwhelming to see what I mean at first: our own framework is pretty cool, too — but it's also quite similar. Both are built on decorators, after all: @click.command() turns a function into a command (actually replacing it with a complex object that parses sys.argv when called), @click.argument and @click.option similarly declare the parameters expected to arrive as that function's parameters, and @click.group() lets us collect subcommands in a similar way to git, where pull and push are subcommands of git. So let's rewrite our calculator, this time with a Click group, an add command and a hello command:

# calc.py
import functools
import operator

import click


@click.group()
def main():
    """A tiny calculator."""


@main.command()
@click.argument('x', type=int)
@click.argument('ys', metavar='Y...', type=int, nargs=-1, required=True)
@click.option('-m', '--multiply', is_flag=True, help='Multiply instead of add.')
def add(x, ys, multiply=False):
    """Sum (or multiply) two integers or more."""
    numbers = [x, *ys]
    if multiply:
        click.echo(functools.reduce(operator.mul, numbers))
    else:
        click.echo(sum(numbers))


@main.command()
@click.option('--name', envvar='NAME', prompt=True)
def hello(name):
    """Say hello."""
    click.echo(f'Hello, {name}!')


if __name__ == '__main__':
    main()

If we compare it to our own framework, it's pretty similar, but there are some interesting differences. x is an int argument; ys is variadic (that is, nargs=-1, which is a bit unlovely, but it's Click's way of saying "any number", while required=True means we expect one or more), and it has a metavar for the help message. -m/--multiply is a flag (that is, is_flag=True, which is nicer to read than store_true); and while the default value in the signature is not necessary (since Click always passes either True or False), it's nice to include for the sake of readers, who might not know that this is a flag without reading the decorators. Finally, we use click.echo instead of print, because it can handle Unicode on Windows consoles and other output quirks. Also, note that in older versions of this example, I used @click.argument('y', 'ys', ...), to be able to refer to ys in the function signature but still display y in the help message; this no longer works in Click, which raises a TypeError complaining about "Arguments take exactly one parameter declaration", so we use a metavar instead. Let's see it in action:

$ python calc.py --help
Usage: calc.py [OPTIONS] COMMAND [ARGS]...

  A tiny calculator.

Options:
  --help  Show this message and exit.

Commands:
  add    Sum (or multiply) two integers or more.
  hello  Say hello.
$ python calc.py add --help
Usage: calc.py add [OPTIONS] X Y...

  Sum (or multiply) two integers or more.

Options:
  -m, --multiply  Multiply instead of add.
  --help          Show this message and exit.
$ python calc.py add 1 2 3
6
$ python calc.py add -m 1 2 3 4 5
120
$ python calc.py add 1 two; echo "exit code: $?"
Usage: calc.py add [OPTIONS] X Y...
Try 'calc.py add --help' for help.

Error: Invalid value for 'Y...': 'two' is not a valid integer.
exit code: 2

As you can see, the help messages are auto-generated from the docstrings and the declarations, both for the group and for each of the commands. If we make a mistake, the error message is pretty informative, telling us exactly which value was invalid, and suggesting --help to get more information. It also exits the program with code 2, which is the same as what argparse does.

The hello command has an option with envvar='NAME' and prompt=True, which means that if --name is not passed, Click will try to get this information from a NAME environment variable, and if that is not set either, it will prompt the user for it on stdin:

$ python calc.py hello --name=Alice
Hello, Alice!
$ echo Alice | python calc.py hello
Name: Hello, Alice!
$ NAME=Alice python calc.py hello
Hello, Alice!
$ python calc.py hi
Usage: calc.py [OPTIONS] COMMAND [ARGS]...
Try 'calc.py --help' for help.

Error: No such command 'hi'.

If we pass the answer through a pipe, it isn't echoed, so Hello, Alice! continues right on the prompt's line; if we typed it manually, we'd see Name: Alice and hit enter. Should you use flags or prompts? Why not both — the flags are there for scripts, and the prompts are there for humans who forgot.

Another interesting feature of Click is its support for colors. click.style is a simple way to wrap text in ANSI escape codes, like \x1b[31m for red and \x1b[0m to reset the color afterwards. But if we forget to add the reset part, the color bleeds over into everything that is printed after it — including our shell prompt. click.echo, on the other hand, is smart enough to strip these codes if the output is not a terminal, so we don't end up with garbage if we redirect the program into a file or pipe it through grep:

$ python -c "print('\x1b[31mERROR\x1b[0m')" | cat -v
^[[31mERROR^[[0m
$ python -c "import click; click.echo(click.style('ERROR', fg='red'))" | cat -v
ERROR

To see it, we need to use the cat -v command, which renders the invisible escape character as ^[ — as you can see, print let the codes through, while click.echo removed them, since our stdout was not a terminal.

Click has many other interesting features, too: click.confirm lets us ask "are you sure? [y/N]", and accepts y, Y, yes and more; click.prompt(hide_input=True) lets us ask for a password without echoing it back to the user; and click.progressbar lets us wrap an iterable and get a progress bar while we iterate over it. All the Sisyphean code we'd have to write ourselves, if we were rolling our own solution.

Groups, Plumbing and Tests

For a slightly more realistic example, consider a simple tool to take notes. Naturally, it has some global options, which we can think of as belonging to the group itself — like the version, but also the file in which we'd like to store our notes (with -f or --file, defaulting to notes.json if unspecified, and respecting the NOTES_FILE environment variable):

# notes.py
import json
import pathlib

import click


@click.group()
@click.version_option('1.0.0')
@click.option('-f', '--file', 'path', type=click.Path(path_type=pathlib.Path),
              default='notes.json', envvar='NOTES_FILE', show_default=True)
@click.pass_context
def main(context, path):
    """Take notes."""
    context.obj = path


def load(path):
    return json.loads(path.read_text()) if path.exists() else []


@main.command()
@click.argument('text')
@click.pass_obj
def add(path, text):
    """Add a note."""
    path.write_text(json.dumps([*load(path), text]))


@main.command('list')
@click.pass_obj
def list_(path):
    """List all notes."""
    for number, note in enumerate(load(path), 1):
        click.echo(f'{number}. {note}')

Let's break it down: the group function, which runs before any subcommand, uses @click.pass_context to gain access to the context object, and store the resolved path in it; while the subcommands, which receive it via @click.pass_obj as their first parameter, can rely on this state having already been prepared. And note that type=click.Path(path_type=pathlib.Path) gives us a pathlib.Path object, not a string.

As an aside, we had to name the list command explicitly, since its function is called list_, to avoid shadowing Python's builtin list. But let's first address a few more interesting topics.

A nice distinction Git makes is that between porcelain and plumbing: the "polished" commands that users are expected to use, like git add or git commit, versus the "low-level" commands that implement them, like git hash-object or git write-tree. This is a useful dichotomy to think about, whether you're designing a CLI or not — it's progressive disclosure once again; people want a wheel, except for some people that actually need the engine. It's OK to have a CLI that serves as plumbing, allowing to dump the database, load it, and clean it up — as well as inspect various things while debugging, or to get a verbose log when something goes wrong.

# notes.py (continued)
@main.group()
def db():
    """Low-level database commands."""


@db.command()
@click.pass_obj
def dump(path):
    """Print the raw database."""
    click.echo(json.dumps(load(path), indent=4))


@db.command()
@click.confirmation_option(prompt='Delete all notes?')
@click.pass_obj
def clean(path):
    """Delete the database."""
    path.unlink(missing_ok=True)


if __name__ == '__main__':
    main()

Note the confirmation_option: it adds a --yes flag, and prompts the user for confirmation if it's omitted — which is a nice safety measure for dangerous operations, ensuring that they aren't called by accident. In general, such operations should always be designed in a way that their side-effects are reversible — but asking first, with the safe answer as the default (that's what the capital N in [y/N] stands for), is a good practice regardless.

$ python notes.py --help
Usage: notes.py [OPTIONS] COMMAND [ARGS]...

  Take notes.

Options:
  --version        Show the version and exit.
  -f, --file PATH  [default: notes.json]
  --help           Show this message and exit.

Commands:
  add   Add a note.
  db    Low-level database commands.
  list  List all notes.
$ python notes.py add 'learn click'
$ python notes.py add 'write a CLI for everything'
$ python notes.py list
1. learn click
2. write a CLI for everything
$ python notes.py db dump
[
    "learn click",
    "write a CLI for everything"
]
$ python notes.py --file=other.json add 'a different notebook'
$ python notes.py list --file=other.json
Usage: notes.py list [OPTIONS]
Try 'notes.py list --help' for help.

Error: No such option '--file'.
$ echo n | python notes.py db clean; echo "exit code: $?"
Delete all notes? [y/N]: Aborted!
exit code: 1
$ python notes.py db clean --yes
$ python notes.py list
$ python notes.py --version
notes.py, version 1.0.0

Now, we see the help message mention the db group among the commands, and the wayward --file option in the list command. This is a pretty classic pitfall: options are local to the command to which they're defined, so we need to put --file before list, not after. Note that declining the confirmation aborts the command, which is why we see the exit code 1; and that passing --yes to accept it skips the question altogether.

But why go through all this hassle, you might ask — why introduce this weird, global context object, instead of simply relying on the function parameters to communicate? Well, just like we don't want to be passing a bunch of global variables to every function call, we don't want to repeat all the global options in every command's function signature. The group's purpose, then, is to set up a shared state for its subcommands — in this case, parsing the path, but in bigger CLIs it might establish a database connection, or configure an API client. The same holds true for subsubcommands, so with nested groups we can progressively enrich this context, with each layer adding to it.

One last thing to cover, before we get to testing, is exit codes: the number that indicates how our program ended, with 0 meaning it succeeded, and anything else indicating it failed. According to Click's conventions, a program can terminate with exit code 1 if it's aborted (like, if the confirmation dialog is declined, or the user hits Control+C to interrupt the flow), or with exit code 2 if it encounters a usage error, like an unknown option or an argument with a bad value.

Now, how would you test such a beast? Click actually comes equipped with a module just for that, click.testing, which provides a CliRunner class that can invoke our command in a predictable, in-process manner, given some arguments and input; and then capture its output and exit code, as well as catching the SystemExit exception for us.

Here's how I test the notes command, using CliRunner and the tmp_path fixture:

# test_notes.py
from click.testing import CliRunner

from notes import main


def notes(tmp_path, *args, input=None):
    path = tmp_path / 'notes.json'
    return CliRunner().invoke(main, ['--file', str(path), *args], input=input)


def test_add_and_list(tmp_path):
    result = notes(tmp_path, 'add', 'hello')
    assert result.exit_code == 0
    result = notes(tmp_path, 'list')
    assert result.output == '1. hello\n'


def test_clean_needs_confirmation(tmp_path):
    notes(tmp_path, 'add', 'hello')
    result = notes(tmp_path, 'db', 'clean', input='n\n')
    assert result.exit_code == 1
    assert 'Aborted!' in result.output
    assert notes(tmp_path, 'list').output == '1. hello\n'

The notes helper simply passes a temporary file via --file, so we never touch the actual notes file; and the second test feeds n into the confirmation prompt, making sure that our note is still there. (As an aside, in older versions of Click you'd see runner.isolated_filesystem(), but this was deprecated in favor of tmp_path.)

Now, if we run the tests:

$ python -m pytest -q test_notes.py
..                                                                       [100%]
2 passed in 0.79s

There's a nice little warning from an old version of this exercise, which I'd like to share. It says that if you ever write tests like this and they pass, make sure to actually look at them and double-check — because if you accidentally forget the assert, such tests will actually pass no matter what (the comparison is computed, and then simply discarded). And that code actually had another bug, in that the ANSI escape sequence should have been \x1b[, not \x31[ — but nobody noticed, exactly because of this. Moral of the story: make sure that your tests actually test something, by seeing them fail at least once.

Anyway, design for CLI is a bit like design for testing: it's that narrow, weird pipe of strings that forces you to consider how someone else might be using your system — in a situation where they don't have access to your types and objects, only this limited interface. As a result, you're often forced to come up with solutions that are much simpler and much prettier.

From Script to Command

If we combine the last two examples, we can do something really cool: generate a command-line interface for any module — in fact, generate it from any module. Because all Click does is provide decorators that are a nice sugar-coating over the underlying objects — click.Command, click.Argument, click.Option, click.Group — we can actually create these objects ourselves, based on a function's signature, and end up with the same result as if we had decorated it ourselves (but without having to do so). Check this out:

# meta.py
import importlib.util
import inspect
import sys

import click


def load(path):
    spec = importlib.util.spec_from_file_location('target', path)
    module = importlib.util.module_from_spec(spec)
    spec.loader.exec_module(module)
    return module


def make_command(function):
    params = []
    for name, param in inspect.signature(function).parameters.items():
        if param.default is inspect.Parameter.empty:
            params.append(click.Argument([name]))
        elif isinstance(param.default, bool):
            params.append(click.Option([f'--{name}'], is_flag=True))
        else:
            params.append(click.Option([f'--{name}'], default=param.default, show_default=True))
    return click.Command(function.__name__, callback=function, params=params, help=function.__doc__)

That's all it takes: load uses the same importlib trick we've seen in the Modules chapter to load a file given its path; and make_command maps each parameter to either an Argument (if there's no default value), or an Option (if the default value is either True or False, making it a flag, or anything else, in which case the help message will reflect this default value). The function's docstring is used for the help message, just like we had before, and that's pretty much it. We can even wrap this functionality in a neat little main function:

# meta.py (continued)
def main(argv):
    if len(argv) < 2:
        print(f'USAGE: {argv[0]} MODULE [COMMAND] [ARGS]', file=sys.stderr)
        return 1
    module = load(argv[1])
    group = click.Group()
    for name, value in vars(module).items():
        if inspect.isfunction(value) and value.__module__ == 'target' and not name.startswith('_'):
            group.add_command(make_command(value))
    group.main(argv[2:], prog_name=f'{argv[0]} {argv[1]}')


if __name__ == '__main__':
    sys.exit(main(sys.argv))

All main does is collect the functions defined in the module (excluding any helpers we might have imported from elsewhere, hence the __module__ check), then add a command for each to a group, which it then runs on the remaining arguments, using a program name that makes sense in the context of this help message. This is actually a bit of a bug fix: the original solution applied the click.argument decorators in the order in which they were defined in the signature, which actually reverses their order (because decorators are applied bottom-up, Click actually stores them in reverse). It then paired the default values with the reversed parameter names, which didn't make sense (and didn't match, either, if there was more than one default value). This way, we actually construct the list of parameters in order, so everything checks out.

So we can do:

# module.py
def add(x, y):
    """Add two numbers."""
    print(int(x) + int(y))


def hello(name='stranger', uppercase=False):
    """Say hello."""
    message = f'Hello, {name}!'
    print(message.upper() if uppercase else message)
$ python meta.py module.py --help
Usage: meta.py module.py [OPTIONS] COMMAND [ARGS]...

Options:
  --help  Show this message and exit.

Commands:
  add    Add two numbers.
  hello  Say hello.
$ python meta.py module.py add 1 2
3
$ python meta.py module.py hello --help
Usage: meta.py module.py hello [OPTIONS]

  Say hello.

Options:
  --name TEXT  [default: stranger]
  --uppercase
  --help       Show this message and exit.
$ python meta.py module.py hello --name=Alice --uppercase
HELLO, ALICE!
$ python meta.py module.py add 1; echo "exit code: $?"
Usage: meta.py module.py add [OPTIONS] X Y
Try 'meta.py module.py add --help' for help.

Error: Missing argument 'Y'.
exit code: 2

And suddenly, we have generated a command-line interface from an API, simply by pointing meta.py at it — with help messages, flags and errors, all for free. It's that simple — and that powerful. If you're familiar with Python's ecosystem, this might remind you of a similar tool, called Fire. What Fire does is exactly what we've been doing — generating a command-line interface from a module's functions — but with much more elaborate introspection. I like Click, though: it's simple enough to wrap your head around, but powerful enough to get the job done.

So, how do we use this? Well, we saw in the Modules chapter that when we run a package as a module — that is, use python -m package — it attempts to execute the package's __main__.py file, which is precisely where such a command-line interface would belong. We could do that — or define an entry point in pyproject.toml, and install it:

$ python -m calc add 1 2 3
6
$ cat pyproject.toml
[project]
name = "calc"
version = "0.1.0"
dependencies = ["click"]

[project.scripts]
calc = "calc.__main__:main"

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
$ python -m venv .env
$ .env/bin/pip install --quiet --disable-pip-version-check --editable .
$ .env/bin/calc add -m 1 2 3 4 5
120
$ tail -n +2 .env/bin/calc
import sys
from calc.__main__ import main
if __name__ == '__main__':
    if sys.argv[0].endswith('.exe'):
        sys.argv[0] = sys.argv[0][:-4]
    sys.exit(main())

Note that python -m calc works even before we've installed anything: using -m makes the current directory part of the import path, so it can find calc/ there and run its __main__.py. What we're doing, then, is defining the entry point for our package; when we install it (and I usually do this in editable mode, -e, so that any changes we make take effect immediately), pip will generate a small, executable script that makes this happen — and the reason it knows what to make happen is because we've specified it in pyproject.toml's [project.scripts] section, mapping the command name to the module and function that will fulfill this promise. In other words, it generates a small script in our environment's bin/ directory, which imports this function and calls sys.exit(main()) — the same as what we've been writing by hand all along (and it's nice to see it done by someone else for a change). That's all pytest is (or pip, for that matter — a callback to our chapter on workflow).

As an aside, there's a lazier way I sometimes use: a script called add-command.sh, which is mentioned in the README file. This little guy just appends alias calc='python -m calc' to ~/.bashrc, and it's pretty amazing how much difference a few good commands can make to the gulf of execution.

One last thing to cover is autocompletion: we all hate typing, and it would be nice if hitting Tab could complete not just paths and filenames, but our own commands and options. In bash, we do this by defining a function and registering it with complete -F function program; this function should then define the COMPREPLY array, usually using compgen — but it's poorly documented, and the best way to figure out how it works is by reading git-completion.bash, which is thousands of lines of such agony. So here's a little Whorfian trick for you: the Sapir-Whorf hypothesis states that the structure of our language determines or at least influences the way we think about the world — whether it's the famous example of how the Eskimos have so many words for snow, or the more general Newspeak from 1984, where the government attempts to reduce the population's ability to think critically by limiting the language they speak. It's not a popular theory, and doesn't really hold true, but it's a fun thought exercise; how would the problem look different if we could phrase it in a different language? Like, what if we could ask bash to run Python, and let it complete the command? The funny thing is, this is exactly what Click does:

$ COMP_WORDS='calc a' COMP_CWORD=1 _CALC_COMPLETE=bash_complete .env/bin/calc
plain,add
$ COMP_WORDS='calc add -' COMP_CWORD=2 _CALC_COMPLETE=bash_complete .env/bin/calc
plain,-m
plain,--multiply
plain,--help

Let's take a look at how it's done: the script that Click generates (which you evaluate from ~/.bashrc with _CALC_COMPLETE=bash_source calc, so that it's loaded on start up — it needs bash 4.4 or higher, but there's a similar zsh_source script as well) calls our program back, this time with _CALC_COMPLETE=bash_complete and the current words; and Click answers with typed completions (e.g. plain,add) based on the same declarations that produce the help. In other words, all bash does is shuttle some strings back and forth, and Click does all the heavy lifting on the other side.

CLIs: Conclusion

One last curiosity, though, and this one's really cool. So far, we've seen that a completion framework requires both a file to source in bash and a program to execute in Python; but what if it could be the same file? I'm talking about a polyglot script, which is valid in more than one language (poly, many; glot, language). Check this out:

$ cat polyglot.txt
''':'
echo 'bash'
exit
'''
print('python')
$ python polyglot.txt
python
$ bash polyglot.txt
bash

If we run it with Python, it prints "python"; and if we run it with bash, it prints "bash"! In Python, the first four lines are considered a string (because of the triple quotes), which we don't use (that's the expression statement which does nothing), so it's discarded; and then we have the print. In bash, though, adjacent quoted strings are concatenated. So effectively, the first line is ''':' — that is, an empty string, followed by ':'; and since : is a no-op command in bash, this line does nothing, so we can treat it as a comment. Then, echo (and an exit, just to avoid bash getting upset over the rest of the Python code, which it doesn't understand — but then again, it never gets there, since it executes line by line, and exits before reaching it).

The reason I'm telling you all this is because I want to hammer home the key lesson of this chapter: a CLI is the honest middle between an API and a GUI. It's easy to build; it's easy to use (at least, by anyone who's not afraid of a terminal; my mother wouldn't touch it with a ten-foot pole, but operations, QA and research personnel would); and most importantly, it's trivially automatable, which means that technical users can build on top of your system, thus turning it from a solution into a platform.

If you think about it, the ideal CLI would be nothing more than a thin wrapper around an API — prog foo 1 is translated into a request to /foo with the argument 1. This means that anything you can do with the CLI, a program can do just as well — which is why it makes sense to develop the CLI first, and the GUI later: the distillation of the core functionality and its various flags into a format that's digestible through the terminal is exactly the analysis you'd have to do for a GUI, so that becomes much easier.

More than that: I'd argue that you should consider building such CLIs not just for your project, but for yourself: whenever you're faced with a task that you have to do more than once, but don't want to do over and over again — the worst kind of tasks, those that are just on the border of being annoying enough to warrant a script, but not critical enough to justify it — a good CLI library might be just the right tool to lower that threshold, and over time those little scripts can snowball into a full-fledged system that's tailored exactly to your needs and preferences (remember the argument about tools that increase our agency from the Workflow chapter?).

In the words of Antoine de Saint-Exupéry (roughly): "if you want to build a ship, don't drum up people together to collect wood and don't assign them tasks and work, but rather teach them to long for the endless immensity of the sea". Some of the stuff in this chapter, like our completion and polyglot scripts, is here exactly for this reason.

Anyway, don't underestimate CLIs — that's how computer science was invented. Use Click, draw crazy ASCII art (with PrettyTable for tables, curses or blessings for full-text interfaces, termgraph for charts) and maybe even add autocompletion, if you're bold enough. But then, there's some data that's inherently visual, and most people expect windows and buttons; so in the next chapter, we'll talk about GUIs, why native ones are overrated, and how the answer is, invariably, Javascript.