Complexity: Where It Comes From, and Who Pays for It

I Know It When I See It

For most of this book, we've been talking about the craft: how Python works, from assignment to GUIs. We've touched on design a few times: we saw startswith's tuple in the chapter on native types, __bool__ and Scott Meyers in our discussion of magic methods, the inject function in Functions 3, and of course the various patterns in the previous chapter. But in this last chapter, I'd like to step back a little, and try to address the question all of those examples have in common: what makes a design good, and where does badness come from?

This question is not unlike Potter Stewart's 1964 description of hardcore pornography: "I know it when I see it," he famously said. That's more or less the state of the art in software design: it's widely agreed that urllib's password manager, handler and opener, which we encountered in our introduction, is not as good as requests' auth= tuple — but it's hard to explain exactly why without some vocabulary. This chapter is an attempt to develop some.

To start with a definition, a design is a plan or specification for the construction of an object or system, or for the implementation of an activity or process. The first thing you might notice about this definition is that it doesn't mention software design, let alone programming; but then, if everything was simple enough to understand at a glance, we wouldn't need a plan to coordinate its construction at all — we'd just do it. In other words, design is, first and foremost, a response to complexity: not in the computational sense of the word, but in the sense of "a lot of different things going on", or "something that's hard to understand and change".

To deal with such complexity, we can either try to eliminate it, which is hard, and requires first figuring out what's important and what's trivial, because a lot of what makes code complex is not actually essential for solving the problem — or, we can try to hide it, by recognizing some repeating pattern and generalizing it into an abstraction: a function, a class, or a module. In my own terminology, engineering is about implementing this abstraction; design is about refining it, so that it's useful, generalizes well, and we can build on top of it.

To illustrate what this might mean, I'd like to use a specimen that's part of Python itself: startswith can actually take multiple prefixes to check at once — but it has to be a tuple, not *args, and this leads to the curious case of the double parentheses:

>>> 'hello world'.startswith('hello')
True
>>> 'hello world'.startswith(('hi', 'hello'))
True
>>> 'hello world'.startswith('hi', 'hello')
TypeError: slice indices must be integers or None or have an __index__ method
>>> 'hello world'.startswith('world', 6)
True
>>> 'hello world'.find('world', 6)
6

How come?

If we look at the tuple, it works; if we look at the two strings, we get a baffling error message about slice indices, because actually, the second argument is start, an offset borrowed from find and count for the sake of symmetry — and it actually does work: startswith('world', 6) is indeed True. A string that doesn't start with "world" can still "start with" it from its middle; this is coherent, if not very useful. For find and count, the start and end arguments do make sense — when searching through a string, it's a pretty standard use case to only consider part of it, without having to copy the whole thing; but for startswith, it's really an edge case nobody needed, ever.

This is not a dig at Python's core developers — they are all very smart people. It's just that the decision to have such a signature for the sake of symmetry occupied the space, so when a much better use case for this space came along — namely, checking for multiple prefixes at once — it had to squeeze in there as a tuple. Every decision has an impact on subsequent decisions, and this is how complexity compounds.

As an aside, this urge for symmetry is a kind of OCD that's pretty common in the exact sciences; I sometimes catch myself choosing variable names with equal length, just so the equal signs line up. Ken Thompson, when asked what he would do differently if he were to implement Unix again, reportedly said that he'd spell creat with an e.

So let's talk about the origins of complexity. We'll start with dependencies, then move on to state and edge cases, and discuss the multipliers of complexity: cognitive load and multipurposeness. Then we'll meet the people on the other side of the interface, and consider Scott Meyers' most important guideline, Le Corbusier's Radiant City, Jane Jacobs's life work, and Dieter Rams's ten principles of good design. We'll conclude, finally, with what I would write today, and why.

Dependencies

So what is it, exactly, that causes complexity? As I count it, there are three things: dependencies, state, and edge cases. These are the three factors that contribute to complexity, but it's also multiplied by what I like to call cognitive load: being exposed to it all at once, as opposed to little by little; multipurposeness: the attempt to tell several different stories at the same time; and sloppiness: things like inconsistent naming, which makes your brain work harder to decipher meaning, the way you might look at a cloud and see different shapes in it — except in this case, there's no meaning, it's just a cloud; but I trust that it's clear enough why it's bad, so let's not dwell on it.

Let's talk about dependencies, first. Imagine you have to organize an event — but you're not the only one involved: the more people participate in the process, the harder it is to coordinate everything, right? That's what dependencies are about: anything that's not part of the piece you're working on, but that this piece relies on: whether it's a package, a service, a database, a file system, another function — something that, if it breaks or disappears, your piece will break, too. It's something that's not under your control, but that you still depend on. These things are unavoidable, of course, but they can either be hard-coded, or injected: explicitly passed in as parameters. If A depends on B and hard-codes it, then if B changes or disappears, A is screwed; but if B is injected, if it's passed into A as an argument, then someone else can pass in something else, and A can work just as well.

To give an example, here's a function from my course that uses DuckDuckGo's JSON API to get the definition of a word:

>>> import requests
>>> import urllib.parse
>>> def find_definition(word):
...     query = f'define {word}'
...     url = 'https://api.duckduckgo.com/?' + urllib.parse.urlencode({'q': query, 'format': 'json'})
...     response = requests.get(url)
...     data = response.json()
...     definition = data['Definition']
...     if not definition:
...         raise ValueError(f'{word!r} is not a word')
...     return definition
>>> find_definition('serendipity')
ValueError: 'serendipity' is not a word

We start with building a query, then urlencoding it into a URL, then using requests to GET it and parse the JSON, then plucking out the Definition field — and finally, if it's empty, raising an error to complain about it. When I was teaching this function, I told my students that the only things that could go wrong were if requests had a bug, or DuckDuckGo changed its API, which was not my fault and not my problem. But if you actually try running this function now, you'll get the message "'serendipity' is not a word" — because the API has, in fact, changed, and now returns an empty Definition for such queries. That's what I mean by dependencies: something that can change under you, so to speak, without a single line of your code changing at all.

In fact, testing this code is a pain, too: since we'd like to avoid hitting the network, we need to mock out requests — but instead, let's remember that a module is just an object, so we can pass it in as an argument, using the actual one as default:

>>> def find_definition(word, requests=requests):
...     query = f'define {word}'
...     url = 'https://api.duckduckgo.com/?' + urllib.parse.urlencode({'q': query, 'format': 'json'})
...     response = requests.get(url)
...     data = response.json()
...     definition = data['Definition']
...     if not definition:
...         raise ValueError(f'{word!r} is not a word')
...     return definition
>>> class FakeResponse:
...     def __init__(self, data):
...         self.data = data
...     def json(self):
...         return self.data
>>> class FakeRequests:
...     def __init__(self, data):
...         self.data = data
...         self.urls = []
...     def get(self, url):
...         self.urls.append(url)
...         return FakeResponse(self.data)
>>> fake = FakeRequests({'Definition': 'the occurrence of events by chance in a happy way'})
>>> find_definition('serendipity', requests=fake)
'the occurrence of events by chance in a happy way'
>>> fake.urls
['https://api.duckduckgo.com/?q=define+serendipity&format=json']
>>> find_definition('serendipity', requests=FakeRequests({'Definition': ''}))
ValueError: 'serendipity' is not a word

This way, when testing, we can pass in a different requests, and it doesn't even need to be all that sophisticated: as long as it has a get method that returns something with a .json() method, we're good to go (that's duck typing for you). This mock implementation simply records all the URLs it was asked to get (so we can make sure the right query was constructed); and fed a fake response with an empty definition, it lets us cover the error path as well. No monkeypatching, no network — just a dependency, injected.

Back in 1952, at the ACM national meeting, David Wheeler (or D. J. Wheeler, as he was known, which is a much cooler name) presented the paper The Use of Sub-Routines in Programmes, about an idea so intuitive that we often forget it actually had to be invented. In it, he makes the point that "all complexities should, if possible, be buried out of sight"; and in his talk The Clean Architecture in Python, Brandon Rhodes argues that this is not exactly the case for I/O: that instead of burying it, we should decouple it.

Let's introduce two new terms: coupling and cohesion. The first one is about how interdependent two components are; the less they depend on each other, the more loosely coupled they are, and the better. The second term is about how much the two components belong together — whether they are about the same "thing", or part of the same semantic field. Generally, the more cohesive the components are, the better — and usually, there's a trade-off between the two. But sometimes, there's no trade-off at all — just plain inertia: we resign to the status quo, to the path of least resistance, to what's known as "the way things are done around here". And almost never is I/O part of the business logic.

So a bad way to refactor find_definition would be to extract its core into a generic call_json_api(url) function, which would just bury the I/O one level down, making find_definition look simpler, but actually introducing a new dependency — on a function that depends on requests. Two questions you should ask yourself when considering this sort of refactor are: is it more cohesive? In this case, not at all: call_json_api has nothing to do with DuckDuckGo. Is it easier to test? Also not really: we just moved the need to patch requests from find_definition to call_json_api. It's true that reusable code is good — but only if you actually intend to reuse it. If you're only writing it because one day, maybe, you might, then you'll usually be wrong. The computer scientist Joe Armstrong had a nice quip about such cases: "You wanted a banana but what you got was a gorilla holding the banana and the entire jungle."

So let's do it differently. Instead of burying the I/O one level down, let's keep it where it is, and extract everything around it. Here's how find_definition would look if we split it into build_url and pluck_definition, isolating the I/O part in the middle:

>>> def find_definition(word):
...     url = build_url(word)
...     data = requests.get(url).json()
...     return pluck_definition(word, data)
>>> def build_url(word):
...     query = f'define {word}'
...     return 'https://api.duckduckgo.com/?' + urllib.parse.urlencode({'q': query, 'format': 'json'})
>>> def pluck_definition(word, data):
...     definition = data['Definition']
...     if not definition:
...         raise ValueError(f'{word!r} is not a word')
...     return definition
>>> build_url('serendipity')
'https://api.duckduckgo.com/?q=define+serendipity&format=json'
>>> pluck_definition('serendipity', {'Definition': 'a happy accident'})
'a happy accident'
>>> pluck_definition('xyzzy', {'Definition': ''})
ValueError: 'xyzzy' is not a word

The I/O part is still there, mind you, but it's contained: in find_definition, which is now just three lines, and on a single level of abstraction ("eminently readable", as Rhodes would say). The two new functions are now pure, as in they don't have side-effects; and they're also very specific to DuckDuckGo, which makes them pretty cohesive — and also pretty trivial to test, since we can just invoke them on some simple values and make sure the results are as expected. The only thing left to test in find_definition is the integration itself, and one test should suffice.

Contexts, Containers and Cycles

This is all nice, but there's a problem: if we're doing this in a large system, and we want every function to have access to the file system, the database, the message queue and the logger, then suddenly we have to pass them all around all the time, and all our signatures grow from two arguments to twenty. What's more, this is a special case of a more general problem, so let's address it first: when do we use parameters, and when do we use global state?

Every module that our function touches is, in effect, global state; so we can argue that by making it a parameter, we're actually decoupling the function from the scope in which it was defined. Global state is generally frowned upon, but what's so bad about it, really, is that it's global; local state is just fine. In fact, self is a dependency, too — it's just injected automatically, which is exactly what makes object-oriented programming so alluring.

We could, for example, define a single context parameter, which holds context.fs, context.db, context.mq and so on; and in tests, we'd just prepare such a context object, full of fakes. This is actually pretty cohesive, in a way — all these things are part of the world that our code operates in, so we can group them together. We wouldn't want to pass in a bunch of arguments that are actually the input to our algorithm, though, because this is not what we mean by "context".

Alternatively, we can invest in a dependency manager, or container — a component whose sole purpose is to construct, compose, and inject. This might seem a bit meta, and people often don't realize that it's part of the problem, because they're focused on actually solving some concrete problem; but it's a good idea to invest in infrastructure, rather than drag the problem around forever.

If you're familiar with pytest, you've probably seen this already: just add tmp_path or capsys to a test's parameters, and it suddenly becomes available ("you define a function with some arguments, and I'll figure out how to invoke it"). It's not a big context object, mind you — it's targeted, and only constructs what's needed for each particular test. We actually implemented its very core in Functions 3, when we saw the inject function; here it is again, having grown a bit:

>>> import inspect
>>> class Container:
...     def __init__(self):
...         self.providers = {}
...         self.instances = {}
...     def provide(self, function):
...         self.providers[function.__name__] = function
...         return function
...     def get(self, name, path=()):
...         if name in path:
...             raise RuntimeError(' -> '.join(path + (name,)))
...         if name not in self.instances:
...             self.instances[name] = self.call(self.providers[name], path + (name,))
...         return self.instances[name]
...     def call(self, function, path=()):
...         names = inspect.signature(function).parameters
...         return function(**{name: self.get(name, path) for name in names})
>>> container = Container()
>>> @container.provide
... def config():
...     return {'db': 'sqlite:///users.db', 'sender': 'noreply@example.com'}
>>> @container.provide
... def db(config):
...     return f'<db {config["db"]}>'
>>> @container.provide
... def mailer(config):
...     return f'<mailer {config["sender"]}>'
>>> def signup(db, mailer):
...     print('signing up with', db, 'and', mailer)
>>> container.call(signup)
signing up with <db sqlite:///users.db> and <mailer noreply@example.com>

Let's go over it: the provide method registers some function, to be associated with its name; the call method inspects another function's parameters, and resolves each one of them; while the get method attempts to build a dependency by calling its associated provider, but does so in a way similar to the call method, recursively — so it can be injected into a provider that requires it. It also caches the dependency, so it's not reconstructed unnecessarily — note how config is built only once, and then shared by both db and mailer. As for the signup function, it doesn't actually construct anything, and we don't actually call it on twenty arguments — this is done by the infrastructure.

Such containers can be implicit, like pytest does (or FastAPI's Depends), or explicit, where we drive the container by a configuration or a schema that specifies how things fit together. In any case, it's important to note that dependencies should form a directed acyclic graph, or DAG: if A depends on B, which depends on A, how do you construct either of them first? It's a chicken-and-egg problem, and if you've ever encountered a circular import in Python, you've seen this complaint before, where it says the module is partially initialized. Suppose that, in a fresh container, we have:

>>> @container.provide
... def logger(db):
...     return f'<logger to {db}>'
>>> @container.provide
... def db(logger):
...     return f'<db logging to {logger}>'
>>> container.call(signup)
RuntimeError: db -> logger -> db

Then, resolving the db depends on resolving a logger, which depends on resolving the db — which we're already resolving, so get knows to complain about such a cycle: db -> logger -> db. The solution, though, is not in writing some clever code, but in rethinking our design: usually, one of these dependencies doesn't actually need the other, or both of them actually need some third thing.

So, besides hard-coding dependencies, or injecting them into our functions, we can actually discover them; that is, have some component that holds a ticket — a name, or a URL — and asks a central place to resolve it. This is exactly what we did in the last chapter, when we defined drivers to pick an appropriate implementation based on the URL scheme. It's not a question that's as relevant for functions, but definitely something to consider when it comes to services.

State

Dependencies are just a subset of a more general problem: state. In all the solutions we saw, we ended up creating an object with a state, whether a context or a container. But is it really bad? Like every tool, it's not so much good or bad, as useful or not useful; it depends on its design, and what you're trying to do with it — if you're trying to make a salad and you decide to use a hammer, maybe just eat the vegetables uncut, yeah? It's not a question of whether a tool is good or bad, but of whether it's cost-effective: does it remove more complexity than it adds? Don't use state unless you absolutely have to, and if you do, try to organize it in a wise manner. For example, a timer is easier to implement with a class (it has a start time, an end time, and an elapsed time), while a temporary directory is nicer as a generator with contextlib.contextmanager than as a class with __enter__ and __exit__ (like we saw in the chapter about the object lifecycle). There's a good talk about this by Jack Diederich, called Stop Writing Classes.

Anyway, here's a story, based on true events, from the army (although, because the domain was classified, we'll have to pretend it was something else). One day, we decided that it's about time we solve the problem of writing files once and for all, with a proper infrastructure. After all, sometimes the data is so bulky that nobody ever looks at it, so we should compress it; but sometimes people want to actually read it with their own eyes, so we shouldn't. Consider the following:

>>> import gzip, os
>>> def write(data, path, compress=False):
...     if compress:
...         fp = gzip.GzipFile(path, 'wb')
...     else:
...         fp = open(path, 'wb')
...     with fp:
...         fp.write(data)
>>> write(b'x' * 1000, '/tmp/x')
>>> os.stat('/tmp/x').st_size
1000
>>> write(b'x' * 1000, '/tmp/x', compress=True)
>>> os.stat('/tmp/x').st_size
31

When you write a thousand times the letter "x", it actually compresses to 31 bytes, most of which are taken by gzip's header and trailer. One day (and this actually happened), we decided that we'd like to change this behavior, and actually compress everything by default. But afterwards, we noticed that some files were being compressed, while others were not; what did all the uncompressed files have in common? They were all JSON:

>>> def write_json(obj, path, compress=False):
...     data = json.dumps(obj).encode()
...     write(data, path, compress)
>>> def write(data, path, compress=True):
...     if compress:
...         fp = gzip.GzipFile(path, 'wb')
...     else:
...         fp = open(path, 'wb')
...     with fp:
...         fp.write(data)
>>> write(b'x' * 1000, '/tmp/x')
>>> os.stat('/tmp/x').st_size
31
>>> write_json({'text': 'x' * 1000}, '/tmp/x')
>>> os.stat('/tmp/x').st_size
1012

When write's default argument was False, it didn't matter: write_json had a default argument of its own, and explicitly passed it in. But when we flipped the default argument to True, the JSON files remained uncompressed (1012 bytes), because write_json didn't change accordingly. The thing is, while nobody meant any harm, we coupled the two functions by trying to coordinate their default arguments: a dependency, hiding in a signature. The way to avoid this, in general, is to pass None — an argument indicating the absence of an argument, rather than a default value to fall back on. This None can then "bubble" all the way down, until it's resolved into a default behavior at the very bottom, in just one place. I picked None because it's the most common singleton in Python, but you could define a private sentinel object, too:

>>> DEFAULT_COMPRESS = False
>>> def write(data, path, compress=None):
...     if compress is None:
...         compress = DEFAULT_COMPRESS
...     if compress:
...         fp = gzip.GzipFile(path, 'wb')
...     else:
...         fp = open(path, 'wb')
...     with fp:
...         fp.write(data)
>>> def write_json(obj, path, compress=None):
...     data = json.dumps(obj).encode()
...     write(data, path, compress)
>>> write_json({'text': 'x' * 1000}, '/tmp/x')
>>> os.stat('/tmp/x').st_size
1012
>>> DEFAULT_COMPRESS = True
>>> write_json({'text': 'x' * 1000}, '/tmp/x')
>>> os.stat('/tmp/x').st_size
42

In this case, flipping a single global variable will flip everything — and write_json doesn't know or care what the default value is. It's still state, yeah? But at least it's state in one place. One student actually suggested that write_json could simply omit the argument when it's None, thus decoupling the two functions — but at the cost of making the code a bit messier. Anyway, what I'm getting at here is that even something as minute as a default argument is actually a form of state, because it indicates that the function depends on an external decision. A function reacting to a notion ("no argument") is cleaner than a function reacting to a value (False). Functions that pass each other values are coupled — functions that pass each other notions are not (or at least, much less so).

This notion of purity is really important, so let me elaborate on it a bit. A pure function is one that goes from input to output, with no side effects and no state — like the f(x) in math; you don't write f(x) and then say "first, it adds one to x, and then it prints it out". Such functions are at the core of functional languages (like Lisp, Scheme, Clojure and, most strictly, Haskell); in Python, you have lambda (which is an expression, so it doesn't even have a place for statements), as well as map, filter, comprehensions and generators. Another major advantage is horizontal scalability: the fact that there's no state means there's no shared state, so there's no waiting, no locks, no congestion. Once something becomes stateless, running one copy of it or a thousand is exactly the same — which is what we saw in the chapters about concurrency and web development, but from the design perspective. It's the difference between wrapping state in an object, which is basically burying it — and pulling the decision out into a single place, where it's explicitly decoupled.

The Long Tail

We all know about the normal distribution — like, how most people are of average height, but some are really short or really tall. But there's another distribution that's almost as omnipresent, even though fewer people know about it; and it might be more relevant to how we solve problems and manage our lives.

In 1896, Vilfredo Pareto noticed that 80% of the land in Italy was owned by 20% of the population; in fact, the story goes that he noticed a similar pattern in his garden, where 80% of the peas came from 20% of the pea pods. This is known as the Pareto principle: 80% of the effects come from 20% of the causes. In 1935, George Zipf, a linguist, demonstrated that word distribution follows a similar power law: the second most popular word will occur approximately half as often as the first one, the third most popular word about a third as often, and so on.

To illustrate this, let's look at the names used in the standard library:

>>> import collections, pathlib, re, sysconfig
>>> names = collections.Counter()
>>> for path in pathlib.Path(sysconfig.get_path('stdlib')).glob('*.py'):
...     names.update(re.findall(r'[A-Za-z_]\w*', path.read_text()))
>>> len(names), names.total()
(26241, 489296)
>>> names.most_common(6)
[('self', 19937), ('the', 11935), ('if', 11559), ('is', 7805), ('a', 7423), ('return', 7309)]
>>> ranked = [count for name, count in names.most_common()]
>>> [ranked[rank - 1] for rank in (1, 10, 100, 1000, 10000)]
[19937, 5304, 619, 57, 4]
>>> top = ranked[:len(ranked) // 5]
>>> sum(top) / names.total()
0.8892633497923548
>>> ranked.count(1)
8300

We have 26,241 distinct names, and 489,296 occurrences overall. The most common name, not surprisingly, is self, but the second most common name is "the", which is not very interesting — it mostly comes from comments and docstrings. But if you look at the other names, you can see that we go from rank 10 to 100 to 1,000 to 10,000, and the number of occurrences drops by roughly an order of magnitude every time, just like a power law dictates. In fact, the top 20% of the names account for 89% of all the occurrences, and a whopping 8,300 names, almost a third of all the names, are only mentioned once — that's the long tail for you.

And it turns out that Zipfian distributions are everywhere. The population of cities, the amount of website traffic, the wear of a carpet (people tend to step in the same places, so the paths they wear become even more worn) — all these follow the same pattern. The reason for this is a reinforcing feedback loop, also known as the snowball effect: the bigger cities have more jobs, so more people move there, so it becomes even bigger; the more popular websites get more links, so the more popular they become; and so on. In other words, a lot of what we consider to be cause and effect is actually a loop — a circular reaction, if you will. As an aside, I'd like to add that it doesn't necessarily mean it's good; I lived in London for a year and a half, and didn't really like it at all.

Anyway, the Pareto principle is an absolutely life-changing revelation. 80% of your users will use 20% of your features — so if you use Word, you probably use bold and italics and maybe headings, but not so much letter spacing. 80% of your revenue will come from 20% of your customers. 80% of the questions in your class will be asked by 20% of your students. And when it comes to time management, 80% of the value you produce will be through 20% of the effort you make — so how productive you are has much more to do with what you do, than how efficiently you do it.

In Programming Perl, Larry Wall says something very similar: that "fourth generation languages" make some things really easy, but others nearly impossible, while "industrial-strength" languages make almost everything equally difficult (I'd say Java). So Perl was designed to make the easy jobs easy, without making the hard jobs impossible; the "common" 80% should take 20% of the effort, and the rest is what we call edge cases.

And the funny part is, most of these edge cases are not as interesting, or even as important, as you might think. Not always, of course — if you're building a pacemaker, you want it to keep working if the person goes through an airport's metal detector, because people do fly on planes occasionally — but if your website crashes because someone sent it a request that's too big, it's not going to kill anyone.

So on the tactical level, you can try to do things first simple, then correct — or as we say, fail fast and bail early: if the number is negative, just return, get out of my function. You can also handle special cases separately and explicitly, at the beginning, instead of accounting for them all the way through your main algorithm — like all those textbook loops that are needlessly complicated to also work for zero. On the strategic level, you can realize that simplicity and usability often trump completeness, and even correctness; Fibonacci heaps are asymptotically wonderful, and universities love teaching them — but the Linux kernel uses red-black trees, because in the real world, constants matter.

In fact, you can even eat your own dog food: if you're developing a product, you can use it yourself — and this way, the edge cases you'll address will be the ones that actually happen. When I worked at Google, I used to run Android P long before its release, and the bugs I'd report were precisely those bugs that were likely to happen — because they actually happened to me.

And it's OK to be wrong, really. The world is built on an improvised, amorphous consensus, created by a bunch of grown-up children that nobody told that they grew up; so errors are generally OK.

Cognitive Load

The first one is cognitive load: all this complexity, all at once. When people see a lot of options, they get scared — like the moment you launch Photoshop, and you're greeted by a billion of them. To counter that, we can group them in an intuitive hierarchy, and organize them by function, not material: imagine a kitchen where glasses are in one drawer and metal utensils in another (which is what urllib's parsing in one module and requesting in another amounts to), instead of cutlery with cutlery and pots with pots. Then, we can put the most common options on top: it's why in most kitchens, cutlery is in the top drawer, and sometimes even in multiple places, because it makes sense. This is known as progressive disclosure: start with the simple stuff, and reveal more as it's needed.

To demonstrate, let's go back to open:

>>> import inspect
>>> inspect.signature(open)
<Signature (file, mode='r', buffering=-1, encoding=None, errors=None, newline=None, closefd=True, opener=None)>

Let's go over it again: file is required, so it's the first parameter — but then, mode comes next, not buffering, encoding or errors, because it's the second most common consideration when opening a file. Then come buffering, encoding, errors and newline, and then closefd and opener, which almost nobody ever uses, so they come last. This means that you can learn to use open in one call — and you'll probably never even hear about openers, until the day you need to use one. Similarly, requests has .json() baked into its response object — but not .yaml(), because it turns out that almost nobody returns YAML data on the internet.

The point is, don't expose the union of all your functionality — only the intersection that's actually relevant. If your interface tries to accommodate every possible use case, you end up with a situation where the easy jobs become hard, because they have to deal with a lot of machinery that is only relevant for the other jobs. In code, we have a hierarchy of simplicities: first variables, then functions, then generators, then classes — and you should only move up the ladder when you really need to. In other words, it's more important for an interface to be simple, than for the implementation.

To give an example, take unittest vs. pytest: unittest requires you to import a module, subclass TestCase, define a method, invoke assertEqual, and trigger its discovery mechanism. Pytest, on the other hand, just wants a function starting with test and an assert-statement. Does unittest actually give you more? No — it's just that pytest's progressive disclosure is better: it's trying to get you up and running with the least friction, and generally, you shouldn't be using classes if a function would suffice. (As an aside, I used to have assertEquals in my old slides, before they removed this alias in Python 3.12.)

One example I hesitated to give, because it's a bit of a humblebrag, is SQLAlchemy: one of the most important libraries in the Python ecosystem, and generally a much better library than mine. Here's how you'd save one user with their 2.x version of the ORM:

>>> from sqlalchemy import create_engine, String
>>> from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column, Session
>>> class Base(DeclarativeBase):
...     pass
>>> class User(Base):
...     __tablename__ = 'user'
...     id: Mapped[int] = mapped_column(primary_key=True)
...     name: Mapped[str] = mapped_column(String(256))
>>> engine = create_engine('sqlite:///db.sqlite3')
>>> Base.metadata.create_all(engine)
>>> with Session(engine) as session:
...     session.add(User(name='Alice'))
...     session.commit()
>>> with engine.connect() as connection:
...     connection.exec_driver_sql('select * from user').fetchall()
[(1, 'Alice')]

First, you have a declarative base class to inherit from; then you have a mapped class with a table name and some columns, which are typed (and also have a primary key, which I forgot to add to my old slides). Then you have an engine (I'm still not sure what an engine is, but it's somehow related to the database), after which you have to create the tables through the base's metadata, and then you can create a session, add the user to it, and commit it. And that's just to save one poor user to SQLite; SQLAlchemy 2.0 is already much cleaner than the version I had in my slides, and it's still half a dozen concepts.

The reason is that SQL is extremely versatile — and SQLAlchemy wanted to support 100% of it, from locks to transactions to batch inserts. At some point, though, they forgot that most people use SQL for the CRUD operations: create, retrieve, update and delete. So consider a toy Database, where we could do:

>>> import sqlite3
>>> class Database:
...     def __init__(self, url):
...         self.connection = sqlite3.connect(url.removeprefix('sqlite:///'))
...     def __getattr__(self, table):
...         return lambda **fields: Row(self.connection, table, fields)
>>> class Row:
...     def __init__(self, connection, table, fields):
...         self.connection, self.table, self.fields = connection, table, fields
...     def save(self):
...         columns = ', '.join(self.fields)
...         marks = ', '.join('?' for _ in self.fields)
...         self.connection.execute(f'create table if not exists {self.table} ({columns})')
...         self.connection.execute(f'insert into {self.table} ({columns}) values ({marks})', tuple(self.fields.values()))
...         self.connection.commit()
>>> db = Database('sqlite:///users.sqlite3')
>>> user = db.user(name='Alice')
>>> user.save()
>>> db.connection.execute('select * from user').fetchall()
[('Alice',)]

All we have is __getattr__, which we covered in the attributes chapter, turning any attribute into a table name and returning a row factory; and then save, which creates the table if necessary, and then inserts the values, using placeholders. It's a toy example, so don't take it too seriously (for example, the table and column names go straight into the SQL, so please don't feed it user input) — but the point is the interface: the easy job is easy, and the hard jobs are still possible, using db.connection, which is just plain sqlite3. Doesn't learning to use SQLAlchemy give you some expertise? In a perfect world, maybe; but in a world where we're not immortal, we have to prioritize.

As for the counter-argument, I'd like to tell you about the story of worse is better, by Richard Gabriel. While he was trying to understand why C and Unix were overtaking Lisp, he came up with two different approaches: the MIT style (do the right thing) vs. the New Jersey style, where the simplicity of the implementation trumps that of the interface. One example he gives is how Unix lets an interrupted system call fail with an error code (EINTR), and the user's program needs to retry it. This is simple to implement — so simple, in fact, that it can be ported everywhere, which is why Unix and C spread like the ultimate computer viruses.

Gabriel, however, also argued both sides: under the pseudonym Nickieben Bourbaki, he later wrote a rebuttal, Worse Is Better Is Worse. I'm sorry to confuse you with these mixed messages, but it's really the only way I know to teach design.

One Thing Well

The second multiplier is multipurposeness: how many different stories are being told at the same time. Let's say you have a monolithic system vs. a modular one: the monolith has fewer interfaces and less access restrictions — everybody can get to everything, like if you'd try to get things done through friends, or by pulling a few strings, or knowing the right person to talk to in the right department. A modular system, on the other hand, has less responsibilities per part, but more coordination — all these interfaces are forms you have to fill out, a bureaucracy, really.

According to the Unix philosophy, we should do one thing and do it well (the single responsibility principle); we've seen how powerful this approach can be in McIlroy's pipeline, way back in the chapter on CLIs. But then, Linux is a monolithic kernel: all compiled into one big piece, so everything in kernel space can reach everything else — exactly what Andrew Tanenbaum was criticizing in his famous 1992 argument with Linus Torvalds. If you really know what you're doing, then it's OK; the point is to aim for a Goldilocks zone, where things are neither too specific nor too general. I'm sorry, but this really is the message.

Another interesting idea is simplicity. The Unix programming interface is literally open, close, read and write (and a few more); and with it, you can describe files, devices, pipes, sockets, /dev/null and /dev/urandom alike. Whenever you have an idea that fits within a few simple terms, it's usually a good sign you've discovered something. In A Philosophy of Software Design, John Ousterhout calls the best modules "deep": a simple interface that hides a lot of functionality behind it, and he uses Unix's file I/O as an example. Modules that don't manage to do so are "shallow": their interfaces are almost as complex as the functionality they provide, like a class that only forwards a call to another, or urllib's manager, handler and opener, which are three different classes doing very little. Python's open, on the other hand, is definitely deep — one function call hides a lot of buffering, encodings, newline translations, and the whole process of working with files on the operating system, really.

The famous quotes come to mind: "simplicity is the ultimate sophistication" (an old Apple slogan, often attributed to Steve Jobs); it takes a lot of work, because it's much easier to add features than to narrow them down to the ones that are truly needed; "perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away" (by the French writer Antoine de Saint-Exupéry); and the quote often attributed to Albert Einstein, "everything should be as simple as possible, but no simpler".

But the truth is, complexity is not always bad. The first computers weren't simple; they were just good enough; and the hydrogen bomb wasn't simple, either, but it was — well, operational. One counter-argument to this was given by the poet Madeline Gins and the artist Shusaku Arakawa, in their Reversible Destiny Foundation. They suggested an architecture against death — homes so uncomfortable on purpose, with bumpy, slanting floors, that the people living in them would constantly be alert and active, rather than complacent, like the humans in WALL-E. In a similar vein, Cas Holman designed the Rigamajig, a building kit for kids made out of big, chunky wooden planks and real, metal bolts. Parents are afraid it's going to fall on their heads — but Cas's response is that we really don't give kids enough credit. They're smart, and they actually want to be respected.

The point is, maybe motivation is more important than value; because it implies latent value. That's why I'm so comfortable with confusing you in a book like this one: if I manage to excite you, you will go on to program more — and that's more valuable to me than anything I could spoon-feed you. If we're talking about products or tools, though, maybe not: when it comes to a house, I'd like it to be comfortable; when it comes to philosophy, it can be confusing (and anyway, they both died, so it didn't quite work).

Easy to Use Correctly

In the introduction, I promised that we'd revisit Scott Meyers' most important design guideline: make interfaces that are easy to use correctly, and hard to use incorrectly. We've already encountered it with __bool__, and mentioned it in our chapter about CLIs — but now, I'd like to talk about it a bit more substantially.

The first part of the guideline is pretty obvious — the second one is much more interesting. The premise is simple: your users are somewhat smart and somewhat motivated. They're willing to read some documentation, try a few examples — in other words, invest some effort. Nobody comes to work and thinks, "I wonder how I can do a particularly lousy job today" — so if they fail, it's more our fault than theirs.

Ironically, this is exactly the opposite of what overdesigners think: the user must be stupid, and doesn't know what they want — but I've created a system that correctly reflects the problem, so the user will just have to learn to use it properly. But nobody cares about the problem in the world; people care about what they can do, and look for the system that will let them solve their problem in the most effortless way.

Python, for example, makes it hard to use things incorrectly in a myriad of small ways. Take sorted, for example — it refuses a positional key argument; or write, which can expect a keyword-only compress:

>>> sorted(['charlie', 'bob', 'alice'], len)
TypeError: sorted expected 1 argument, got 2
>>> sorted(['charlie', 'bob', 'alice'], key=len)
['bob', 'alice', 'charlie']
>>> def write(data, path, *, compress=None):
...     ...
>>> write(b'x' * 1000, '/tmp/x', True)
TypeError: write() takes 2 positional arguments but 3 were given
>>> write(b'x' * 1000, '/tmp/x', compress=True)

In the first example, we saw that sorted(names, len) doesn't work, because key is a keyword-only argument — so every call looks exactly like what it does, and there's no room for confusion. In the second example, we had a bare *, so compress was also a keyword-only argument, and we couldn't have written the rather opaque write(data, path, True), but had to spell out what the True actually meant — that is, compress=True, which effectively documents itself.

Then, there are Norman doors: doors with a design so bad, that it's not clear from just looking at them whether you're supposed to push or pull. They're named after Don Norman, author of The Design of Everyday Things; and in this case, there's a simple enough solution — a flat plate clearly says "push", and a handle clearly says "pull". So the problem was solved, and all we had to do is stick to the solution — and yet, we keep getting it wrong. Once you notice it, you'll find such doors everywhere; and code has them, too. A method that's called get but actually changes things, or urllib's getcode(), when it's a method call for something that should be an immediate attribute, like requests's status_code.

Or, take Japanese toilets — they are highly sophisticated, but as a tourist (that is, a person who's used toilets most of his life) I felt a little humiliated at not being able to figure out how to operate them. So a startup idea: progressive disclosure for toilets — just a few buttons, but an advanced panel that slides out on demand.

But now for the critique I mentioned in the introduction: this guideline is entirely apolitical. It never asks whether the interface should be easy to use at all — it assumes the user's goal as a given, and focuses on our role as interface designers, which is to remove as much friction from the process as possible. But whether the goal is to file taxes, scroll endlessly, or fire a weapon is not considered; in some cases, it's the friction that's the point — a confirmation before deleting a production database, a rate limit, or a dangerous capability made deliberately hard to reach. "Hard to use incorrectly" presupposes that we know what "incorrectly" means — but it's actually a question of values, not of interfaces: easy for whom, to do what, and at whose expense?

And sometimes, the users' "incorrect" usage is actually the correct one, and it's the interface that's wrong. This is where architecture comes in: it's a field where people quite literally live inside the interfaces.

The Radiant City

The introduction to this book promised blind utopias, and that we'd see how they always fail — so here's the first. In 1930, Le Corbusier proposed the Ville Radieuse, or the Radiant City. His intention was right: cities that were designed using old methods for an old world should be adapted for the modern era; and he proposed a grid of cross-shaped residential towers, with a small footprint but lots of people, who'd get three sides with air and light, as well as separate zones for living, working and recreation, all connected by super-fast transportation. But then, he got very excited about cars: there are broad highways that cut right through the city, and in the original plans its main street had seven lanes (in his previous plan for a contemporary city, he even put an airport right above the central train station). From above, it looks futuristic; but at street level, it looks like the projects.

This started a trend: Lúcio Costa, when he designed the city of Brasília, the new capital of Brazil (it was selected in 1957, and inaugurated in 1960), similarly separated the commercial, residential and recreational zones. And while this might sound good in theory, in practice it means that everyone goes to the same zone in the morning, so the traffic is terrible. There's no spontaneity, no serendipity — no lucky accident of an unlikely collaboration; the park is just far enough that going there is a plan, not a whim, and the residential streets, without any shops or cafés, are dark and empty.

In New York, the same happened with the Lower Manhattan Expressway, pushed forward by Robert Moses, who for decades had more power over the city's building than any elected official. It was an elevated highway right across the island, from the Holland Tunnel to the East River bridges, which would raze much of SoHo and Little Italy in the name of slum clearance and urban renewal.

And then, Jane Jacobs happened. She was a journalist with no credentials in urban planning, but she led the opposition against the project; and in 1962, the city shelved the expressway. It was revived, but eventually killed for good by the end of the decade — a little, old grandmother of a journalist beat the most connected man in New York. In her book, The Death and Life of Great American Cities, published in 1961, she wrote that sidewalks promote safety and contact: the more eyes there are on the street, the safer it is, so the more people want to go out, and the more eyes there are on the street. It encourages creativity and serendipity, and it's even safer and more fun for children — patrolling empty streets mostly makes people tense, which is why Tel Aviv is not scary at night, no matter where you go, because there are lots of people everywhere. She has a great quote, too: "the myth that playgrounds and grass and hired guards or supervisors are innately wholesome for children and that city streets, filled with ordinary people, are innately evil for children, boils down to a deep contempt for ordinary people." — what the urban planners didn't like was, quite simply, ordinary people.

Ramot Polin in Jerusalem is another good example: it's a honeycomb of five-sided concrete cells from the 70s, designed by Zvi Hecker, very innovative and very unusual. Today, its residents have added all sorts of boxy extensions, air-conditioners and laundry lines, and it looks like a monster (one of my students said it looks like the Borg). They needed somewhere to hang their laundry, somewhere to cool down (it's in Israel, after all), and more space, as their families grew. And can you really blame them, side with the architect and tell them that they didn't use the building correctly? Isn't it more reasonable to say that the architect should have designed it with the values of the people who would live there in mind? In Scott Meyers' words, these buildings were very easy to use incorrectly — and as he points out, users are smart and motivated, even more so when it comes to how they want to live their lives.

The built environment is the ultimate platform, and it really does shape us. When I lived in London for a year and a half, it was in a residential neighbourhood; and it was so cold and dark, with everyone indoors, and the same handsome Victorian facades repeating over and over again, that I felt small and lost and unimportant. In Tel Aviv, with all its cafés and vintage stores and hummus places, all its people laughing and talking, I feel safe, like I belong; and of course, the weather helps.

Solutions and Platforms

The second utopia from our introduction is somewhat briefer: in 1996, John Perry Barlow wrote A Declaration of the Independence of Cyberspace, and addressed the governments of the industrial world with "you have no sovereignty where we gather". It was a beautiful vision: a radiant city of the mind, governing itself by consensus; except that within two decades, much of it was fenced off by a handful of platforms. And again, the same blindness: a plan for people, which assumes who will come and what they will want.

Software, too, has its own radiant cities: grand frameworks, meticulously designed top-down, and then bestowed upon us from above, according to the architect's model of the world. It's beautiful to behold, until you realize it's full of "residents" hanging their laundry between the buildings: monkeypatches, forks, wrappers and workarounds. Before blaming the users, though, we should ask what they needed.

Solutions solve particular, familiar problems, while platforms enable and facilitate solutions, help adapt existing ones and come up with new ones. Which one to build is really a question of how well you can predict the future. A startup operating in a market that's still forming will find out that any particular solution they might have developed will no longer apply, as soon as the market matures; so they'd better develop a platform. A solution is OK for a market we already understand.

In fact, that's what prescriptive (or, if we're honest, paternalistic) theories provide us with: solutions — the output of the author's evaluation function, computed on the author's context. But if we open-source the function itself, we can let others run it on different arguments: different social contexts, different values, or their own future. I preserved the value, not produced it once, so that others can access it — and that's what I've tried to do in this book, too: not just tell you what to do, but explain why.

Take EVE Online, for example: instead of providing you with a series of quests with a fixed route, they actually built infrastructure for trading, manufacturing and fighting — and the players built an economy, corporations and alliances, and even politics to match. The company even hired an economist, and published economic reports! An adventure game is a solution; a platform created a world.

Or, consider the story of waterfall vs. agile: the former method, which dominated the software industry for a while, starts with requirements, moves to specifications, then to design, implementation, testing and finally deployment. Like a waterfall, once you've reached a certain point, there's no going back; and if there was a miscommunication with the clients, it's going to be very expensive to fix. Even if everything goes right, by the time the solution is ready, the clients might have moved on, so even right solutions can be obsolete. Agile, on the other hand, works in quick, short iterations, aiming for a minimum viable product, and using a data-driven approach to see if it works, and if not, how it can be improved. It's especially well-suited for software, because changing software is only a matter of rewriting some code, unlike, say, rebuilding a whole wing in a building.

My personal rule is, that whenever I understand the problem, I use a suitable framework to solve it — but whenever I don't, I develop a framework for solving problems like it. Anything in-between is probably a bad idea, which is why I ask all those questions about any design: can the user change the backend? Add a hook? Customize the system in any way? (Think of Hatch's plugins in the Workflow chapter, or the drivers and middleware from the previous one.)

And please, don't try to protect people from themselves. Let them make their own decisions. If anything, have empathy for other people and their values. I grew up as a privileged Ashkenazi kid in Ramat Aviv, and when I learned about the history of the Mizrahi Jews in Israel, I finally understood why certain neighborhoods look the way they do, and why something that seemed wrong to me before, actually looks different now.

Just like in urban design, where the shift was towards more human-centric solutions — with walkable neighborhoods, fewer cars, and cities that encourage mental and physical health — so in software design, and design in general, we can shift towards more human-centric solutions as well.

Less, but Better

In Danish, the word for design is formgivning — giving form. Dieter Rams, a German designer who designed Braun products for decades, wrote ten principles of good design. Let's go over them, as a thought exercise, and see if they apply to software — and what can we learn if they do.

Good design is unobtrusive. If you run ls -a in your home directory, you'll probably get scared by the sheer number of rc files that programs have left there; or when you install something, it goes and creates folders on your desktop. Whenever I run Homebrew, it says "Updating Homebrew..." and takes a while, even when the only thing it ends up telling me is that my package name has a typo (and you can turn it off by setting HOMEBREW_NO_AUTO_UPDATE).

Good design makes a product useful. No duh. But what's useful is not always so obvious; sometimes we get so excited about our solution that we forget to ask whether the problem it solves is all that important, and whether generalizing it is not just a form of procrastination. And when inheriting code, we might be tempted to rewrite it, just to make it our own — but the old, ugly code probably encodes edge cases we haven't considered, as explained by Joel Spolsky in his excellent essay on Netscape's rewrite, Things You Should Never Do.

Good design is aesthetic. The word has to do with the appreciation of beauty — and it's people who do the appreciating. A product should be fun to use, and can even have a character and a story of its own; we do, after all, personify things (I happen to love Javascript, a terrible language, in part because it's so weird it feels like a creature).

Good design makes a product understandable. It's intuitive — the principle of least astonishment — digestible — the principle of progressive disclosure — and relatable — the principle of human-centered design. Good design is also honest — that is, clear; easy to use correctly, and hard to use incorrectly (don't build a Norman door). But also humble — don't make promises you can't keep, don't talk a lot when you have nothing to say.

Good design is long-lasting. This is not just about the quality of the finish, but about maintenance: a system that people want to, and can, maintain is a system that lasts, like Unix and C, half a century later. Kurt Vonnegut has a wonderful quote: "Another flaw in the human character is that everybody wants to build and nobody wants to do maintenance."

Good design is thorough down to the last detail. What about Pareto? But actually, really good design is closer to 100% than to 80%, even if sometimes worse is better. Another way of looking at this is that good design cares about the peripherals — the logging, error handling and monitoring; anything that's not business logic, but just as important. Programmers love a system whose errors guide them straight to the bug.

Good design is innovative. This is actually not quite true — consider the principle of least astonishment again, where a car whose steering wheel is a joystick, or whose red light means "go", would confuse everyone. A more accurate phrasing would be that good design goes hand in hand with innovation; don't go redesigning the past, unless you have a good reason — in which case, you're actually innovating.

Good design is environmentally friendly. In software, that would mean sustainable and cost-efficient; not using resources you don't need — so in other words, good design is performant (or at least, designed to be optimized on demand, like we saw in the performance chapter); a smarter algorithm can let a startup run on one machine, instead of autoscaling to five hundred.

Finally, good design is as little design as possible. In Rams' own words, "less, but better" — concentrating on the essence, back to purity, back to simplicity.

Now, I know rules like these sound trivial, even condescending; but they're good in that they draw your attention to something. Ludwig Wittgenstein once asked his friend: "tell me, why do people always say, it was natural for man to assume that the sun went round the earth rather than that the earth was rotating?" His friend replied: "Well, obviously, because it just looks as though the sun is going round the earth." To which Wittgenstein answered: "well, what would it have looked like if it had looked as though the earth was rotating?"

What I'd Write Today

Let's recap: complexity comes from dependencies (try to make it a DAG, inject it if you can, invest in a manager if it's truly complex), state (use it, but not lightly; better to decouple than to bury) and edge cases (first simple, then correct; make the simple case simple, fail fast and dogfood). It's amplified by cognitive load (progressive disclosure; sometimes worse is better) and multipurposeness (Goldilocks zone). In any case, strive for simplicity, but not oversimplification; complexity is not always bad.

Some litmus tests: does it increase cohesion? Make testing easier? What assumptions are hard-coded, and can (or should) be passed in as arguments? Does it actually need state, or can it be a generator, or even just a function or a value? Write a quickstart, and see how the English differs from the Python — how much simpler can you make the simple case, and how common is it? Describe what it does in one sentence — in a docstring, maybe — and if you find yourself adding "and"s and "but if"s, consider splitting it up. But also, take a step back — and look at the problem from a different angle.

Still, design is a soft skill; and like most soft skills, it's mostly a matter of experience. Here's a story: when I was doing my mandatory military service, I joined a technical intelligence unit. As a newcomer, I was sure I was going to become a cool hacker — so the first assignment they gave me was a bit underwhelming. Write a program that prints "hello world" in C, they said. Piece of cake. But then, the critique: why is your main function void, and doesn't return a value? Well, because — and then, why does it return 0, as if everything is OK? What if your printf fails (wait, what? Can it even fail?)? And if it can, why are you so lenient, and just compare the returned value to the length of your message? What if it actually tries to write it, and just gives up in the middle? So maybe you should put it in a loop and try again, right? And if it gets stuck in an infinite loop, maybe add a counter, so you give up after a thousand times — and why not return a different error code for a system error and for giving up after too many attempts? Eventually, you end up with this:

// hello.c
#include <stdio.h>
#include <string.h>

#define MESSAGE "Hello, world!\n"
#define MAX_ATTEMPTS 1000
#define SUCCESS 0
#define SYSTEM_ERROR 1
#define TOO_MANY_ATTEMPTS 2

int main() {
    int result, printed = 0, to_print = strlen(MESSAGE), attempts = 0;
    while (printed < to_print && attempts++ < MAX_ATTEMPTS) {
        result = printf("%s", &MESSAGE[printed]);
        if (result < 0) {
            return SYSTEM_ERROR;
        }
        printed += result;
    }
    return printed < to_print ? TOO_MANY_ATTEMPTS : SUCCESS;
}

So, it keeps printing from whatever offset it reached, and gives up after a thousand times, returning different error codes for system error and too many attempts. It's probably the most complex hello world program you've ever seen. And it's compiled and run like so:

$ cc hello.c -o hello
$ ./hello; echo $?
Hello, world!
0
$ ./hello >&-; echo $?
0

But if you close the stdout stream, it still exits with 0! That's because printf is not actually writing anything to the output, it just writes to a buffer and reports success; the real write happens when the buffer is flushed, which happens on exit — and that write fails, but nobody checks. There's no bottom to the rabbit hole of edge cases.

If I were to write this program today, I'd probably go for the first version — void main and a single printf — but for completely different reasons. When I first wrote it, I didn't know any better; now, I have an opinion on software design, and that opinion says that the risk is fine for a hello world. It's a result of practice, of experience — and it's from a place that's hard to put into words, so all I can do is tell you a story about it. Peter Norvig has a nice essay about this: Teach Yourself Programming in Ten Years.

Fred Brooks, in his book The Mythical Man-Month (1975), has a name for this: he calls it the second-system effect, and describes how an architect's first work is apt to be spare and clean, because he knows he doesn't know what he's doing; all the frills that occur to him are stored away to be used "next time". "This second is the most dangerous system a man ever designs", he writes: it's over-designed, incorporating all the ideas that were cautiously sidetracked in the first. I identify with this very much: I think it's better to get to your second system as soon as possible, so you can get it out of your system. You will make mistakes — but that's OK.

Good design is hard; and we've covered a lot of contradictory advice: interfaces matter more than implementations, but worse is better; inject everything, but contexts are cumbersome. Maturity is the ability to reconcile contradictory ideas, wholeheartedly. There's a nice quote, often attributed to Emerson: "I cannot remember the books I've read any more than the meals I have eaten; even so, they have made me." — so forget the rules, but remember what they made of you.

Everything we've covered in this book has assumed that code is written and read by humans: that cognitive load is a person's working memory, that a quickstart has a human reader, that empathy is owed to a human user. As I was finishing this book, this assumption started to change, and with it — my plans for what will come next. But that's a story for the epilogue.