Assignment: Names, Values and Scopes

Let's start at the very beginning, with assignment.

And I know, I know — you've probably assigned a variable or two in your life, so this might sound trivial. But I really invite you to keep reading and see for yourself, regardless of your experience, whether you learn something new.

We'll cover the syntax, look into data destructuring, and learn how names, values and scopes work, even in confusing cases like late binding. In fact, you can use this chapter as a litmus test for the rest of the book — a topic might sound basic or familiar, but there's usually a surprising amount of nuance to discuss.

Of course, the core idea behind assignment is very simple — we bind a name to a value, like x to 1; and when we reference that name, it resolves into that value:

>>> x = 1
>>> x
1

But already on this level, before we get into scopes and such, there are some interesting details to address.

Packing and Unpacking

First, there's auto-packing — when we assign multiple values to a single name, Python automatically groups them into a tuple:

>>> numbers = 1, 2, 3
>>> numbers
(1, 2, 3)

Then, there's auto-unpacking — when we assign a single value into multiple names, Python iterates through this value and assigns each of its items to the corresponding name:

>>> x, y, z = numbers
>>> x
1
>>> y
2
>>> z
3

Notice how this means that the value has to be an iterable, and its length has to match the number of names in the assignment. Otherwise, we get an error:

>>> x, y, z = 1
TypeError: cannot unpack non-iterable int object

>>> x, y = numbers
ValueError: too many values to unpack

>>> x, y, z, w = numbers
ValueError: not enough values to unpack

And this behavior is not limited to tuples — we can similarly assign from a list:

>>> x, y, z = [1, 2, 3]
>>> x
1
>>> y
2
>>> z
3

Or even from a string:

>>> x, y, z = 'abc'
>>> x
'a'
>>> y
'b'
>>> z
'c'

Like I said, an iterable — anything that can be iterated over.

Wrangling with a Singleton Set

This can already prove useful in some unexpected ways. Suppose we have a set with just one item, and we want to assign it to a name:

>>> s = {1}
>>> x = ?

We can't index it, because sets have no order:

>>> x = s[0]
TypeError: 'set' object is not subscriptable

And while we can use a method like pop, which would successfully extract the value:

>>> x = s.pop()
>>> x
1

This would also remove it from the set, which is not necessarily what we want:

>>> len(s)
0

Alternatively, we could convert the (original) set into a list, and index that:

>>> x = list(s)[0]
>>> x
1

And it works, but is neither elegant nor efficient. So instead, we can add a single comma:

>>> x, = s

This tells Python that it's not a simple assignment of a single value, but the unpacking of an iterable with a single item — which lets us capture it, with no conversions and no side effects:

>>> x
1

Nested Unpacking

This nifty syntax also works for unpacking iterables nested within the one being assigned. Suppose we have:

>>> numbers = 1, [2, 3]

If we simply try to unpack it into x, y and z, we get an error:

>>> x, y, z = numbers
ValueError: not enough values to unpack

And rightly so — numbers only contains two items: 1, and then a list (with 2 and 3). To communicate this to Python, we need to add parentheses around y and z, indicating that the second item is actually an iterable with two items of its own, so we don't actually mean to assign it as is, but to further unpack it:

>>> x, (y, z) = numbers
>>> x
1
>>> y
2
>>> z
3

As an aside, when we have a nested iterable with a single item:

>>> numbers = 1, [2, [3]]

Unpacking it this way gets a bit clunky — it requires a set of parentheses with a trailing comma that's easy to miss, and generally looks weird:

>>> x, (y, (z,)) = numbers

So instead, we can use square brackets. They work exactly the same, but end up looking a bit tidier:

>>> x, (y, [z]) = numbers

In which case, it would make sense to switch to square brackets entirely, so as to avoid a needlessly confusing inconsistency:

>>> x, [y, [z]] = numbers

We can even add such brackets (or parentheses) to the outermost level of the assignment:

>>> [x, [y, [z]]] = numbers

And while in most cases it's unnecessary clutter, it can be useful when we're unpacking an iterable into many variables with long names, and want to split our code over multiple lines:

>>> numbers = 1, 2, 3
>>> [
...     variable_with_long_name_1,
...     variable_with_long_name_2,
...     variable_with_long_name_3,
... ] = numbers

This would also work for our previous set challenge, by the way:

>>> s = {1}
>>> [x] = s
>>> x
1

Although in this case, I'd argue that the square brackets look a bit unusual, and a trailing comma is somewhat cleaner:

>>> x, = s
>>> x
1

Parallel Assignment

When we put auto-packing and auto-unpacking together, we can get them working in parallel, in that elegant syntax of:

>>> x, y, z = 1, 2, 3

In fact, there's nothing new or special going on. 1, 2 and 3 are simply packed, and then unpacked into x, y and z, just like you would expect.

This is also what makes multiple return values possible. When a function, like divmod, returns more than one value, like both the quotient and the remainder of two numbers:

>>> def divmod(x, y):
...     return x // y, x % y

We can capture those values separately:

>>> q, r = divmod(5, 2)
>>> q
2
>>> r
1

But again, there's nothing new or special about it. The function simply returns the values in a packed tuple, which we either unpack immediately, or not:

>>> divmod(5, 2)
(2, 1)

The same happens when we iterate over values that are themselves iterable, like the items of a dictionary:

>>> d = {'x': 1, 'y': 2}
>>> for item in d.items():
...     print(item[0], item[1])
x 1
y 2

The syntax between the for and in keywords is the same as the left side of an assignment, so we can unpack the item into a key and a value at once:

>>> for key, value in d.items():
...     print(key, value)
x 1
y 2

That is, because dictionary items are nothing more than a sequence of tuples (and more specifically, pairs):

>>> list(d.items())
[('x', 1), ('y', 2)]

This syntax also lets us write Fibonacci's function in a very concise way. It's often used to illustrate Python's elegance, especially if you're coming over from languages like Java or C. It returns the th number in Fiboniacci's sequence, where each number is the sum of the previous two, starting from and :

>>> def fib(n):
...     a, b = 0, 1
...     for i in range(n):
...         a, b = b, a + b
...     return a

Since b and a + b are evaluated and packed first, we can unpack and (re)assign the result into a and b, without the need for temporary variables or other swapping tricks.

The Design of Assignment

All of this might seem like unimportant, aesthetic details, but it's just the surface of a much more profound lesson in software design.

Like we said, at its core, assignment serves a very practical purpose. It lets us define or compute a value once, and then reuse it multiple times. This way we can keep state, remove code duplication, increase efficiency and so on; it's great, simple and useful, which is why we have it in virtually every programming language.

But for some reason, Python (and a few other languages, like Javascript) have all this additional, elaborate syntax. Was it really added just for the sake of nice-to-have, cosmetic touch ups?

Consider how assignment also serves an informative purpose. It lets us call a piece of data by a meaningful name, which makes the rest of our code more intelligible and concise.

For example, if we have to print all the keys with positive values in a dictionary, it might end up looking quite opaque:

>>> for item in d.items():
...     if item[1] > 0:
...         print(item[0])

To make it clearer, we can break each item into its parts, giving them explicit and expressive names:

>>> for item in d.items():
...     key = item[0]
...     value = item[1]
...     if key > 0:
...         print(value)

But this makes our code more tedious and harder to follow, especially if an item has many parts. Luckily, as we've seen, we can unpack it already in the loop's header:

>>> for key, value in d.items():
...     if key > 0:
...         print(value)

And it makes sense, doesn't it? It coheres — it's strangley intuitive and satisfying.

These are generally indicators of good design, so it's worth asking what exactly about it works so well.

Data Destructuring

Here's a suggestion: languages like Python and Javascript come equipped with native data structures, like lists and dictionaries. I'd even argue that it's one of their key strengths — the availability of complex containers makes it much easier to organize and manipulate data without the overhead of defining custom objects. Not surprisingly, this is one of the first features we learn about in these languages, and one we end up using all the time.

But if you think about it, it's just one side of the coin. We have a convenient syntax that lets us structure data in complex ways... but after we pass it around, we need to destructure this data: to reference or extract its particular parts.

If we want to keep our code tidy, it's important to organize our information well — but it's also important to retrieve it well. This is almost never addressed; maybe because assignment is often considered such a "basic" topic that it's rarely discussed in depth.

Without good destructuring, though, a lot of otherwise high quality code ends up needlessly cumbersome and brittle. If we spend another moment's thought on assignment, it doesn't just make our code more beautiful — it also makes it more robust and explicit, because it adds a certain assertion about the data's shape.

For example, suppose we have a list of pairs (at least, allegedly):

>>> pairs = [(1, 2), (3, 4), (5, 6, 7)]

To extract the th pair into x and y, we can do:

>>> x = pairs[i][0]
>>> y = pairs[i][1]

Not only is this clumsy, but it would silently pass even when a pair is corrupted, and actually has three values, like (5, 6, 7). If we specify the exact layout, however:

>>> x, y = pairs[i]

Not only is it prettier, but also safer and more expressive — it effectively requires that the th pair has exactly two items.

Star Capturing

Of course, this can be desirable, or not — safety is restrictive by nature, so it's a common trade-off between security and usability.

What if our data actually is variable, and might contain pairs, triplets, and more? Should we revert back to fragile indexing?

According to our newly established logic, one of the key advantages of Python is that it lets us easily create flexible data structures, and it should be equally easy to destructure them.

We can create lists with multiple items — and conversely, we can assign multiple items from a list. We can nest lists — and we have nested unpacking. We can create lists of variable sizes — and as it turns out, we can destructure a list of a variable size, using extended iterable unpacking. I'm going to call it star capturing, because it uses the star operator, or asterisk (*), to... well, capture stuff.

Suppose we want to unpack the numbers 1 to 5 into just three names, x, y and "the rest":

>>> numbers = 1, 2, 3, 4, 5
>>> x, y, rest = ?

Regular unpacking is not going to work. But if we add a star next to rest, Python will know to put any remaining values there:

>>> x, y, *rest = numbers
>>> x
1
>>> y
2
>>> rest
[3, 4, 5]

We can also move it to the beginning, in which case Python will match the last names, and know to put any preceding values in rest:

>>> *rest, x, y = numbers
>>> x
4
>>> y
5
>>> rest
[1, 2, 3]

We can even move it to the middle, and Python is smart enough to understand that the first value goes into x , the last value goes into y, and rest is left with the values in between:

>>> x, *rest, y = numbers
>>> x
1
>>> y
5
>>> rest
[2, 3, 4]

We can put the star anywhere, really, so long as it's deterministic.

For example, using more than one star in an assignment will usually fail, because it's ambigious — it's unclear which values go where:

>>> *x, *y = numbers
SyntaxError: multiple starred expression in assignment

With this trick, we can get back to our list of pairs, and extract the first two values of anything that's a pair, a triplet or more:

>>> x, y, *_ = pairs[i]

Any surplus goes into the underscore (_) — a name that, by convention, means "whatever" or "don't care", so it's often used to absorb irrelevant information.

To be clear, this is still more expressive than plain indexing, because it still asserts what little we do know about our data's shape — namely, that it should have at least two items.

We could take it one step further, and unpack the first pair with no indexing at all:

>>> (x, y, *_), *_ = pairs

And notice how we're actually using the star twice, but because of the nesting it's not actually ambiguous.

However, in this case, I'd argue that it's excessive. It doesn't really add any substantial information about the data's shape — just that it's an iterable with at least one item, which using plain indexing implies as well.

Maybe in cases where indexing doesn't work, like with a set of pairs, we would resort to this arrangement; but in any case, depending on the structure we're trying to express, being able to use ranges is pretty cool.

Exploding Stars

So next to a name, the star operator captures "the rest" of the items. When it's on the other side of the assignment, next to a value, it works differently. I'm going to call it star exploding, because it explodes a sequence into its items, breaking it down into its constituent parts.

Suppose we want to concatenate one sequence of numbers to another:

>>> x = 4, 5
>>> numbers = 1, 2, 3, x

If we use x as is, we end up with a sequence of four numbers, not five; or rather, four items, since the last one is not actually a number, is it?

>>> len(numbers)
4
>>> numbers
(1, 2, 3, (4, 5))

But if we add a star next to the x, it tells Python to spread it, effectively flattening it into its context — the actual x[0], x[1] and so on, until x[n]; or in our case, the actual numbers, 4 and 5:

>>> numbers = 1, 2, 3, *x
>>> len(numbers)
5
>>> numbers
(1, 2, 3, 4, 5)

Note that in this case, there's never any ambiguity — we can have as many exploding stars as we want. If we take two sequences, x and y, and just put them in z, unsurprisingly we end up with a tuple of two items, both of which are tuples themselves:

>>> x = 1, 2, 3
>>> y = 4, 5
>>> z = x, y
>>> len(z)
2
>>> z
((1, 2, 3), (4, 5))

But we if add stars, we reduce those tuples into their actual values, successfully concatenating them:

>>> z = *x, *y
>>> len(z)
5
>>> z
(1, 2, 3, 4, 5)

And notice the nice symmetry in the syntax: the star inherently deals with ranges, but on the unpacking side it means capturing a range, and on the packing side it means exploding it.

Reveries of Dictionaries

We could imagine taking this logic one step further, to destructure dictionaries. Python doesn't currently support this, but I think it demonstrates the point of how any structuring can be mirrored by destructuring, so assignment effectively becomes a query language into our variables.

How would it look if we could fetch specific keys by name — say, by using curly braces?

>>> d = {'x': 1, 'y': 2}
>>> {x, y} = d
>>> x
1
>>> y
2

Javascript actually supports such syntax, you know. And don't think about it as just some syntactic sugar — this would also assert that the keys exist, and generally combine with nesting to support access to a whole other level of complex data structures. Maybe something like this:

>>> d = [{'x': [1]}]
>>> {'x': [x]}, = d
>>> x
1

I don't know; like I said, this syntax doesn't actually work — the point is that assignment is much more interesting and powerful than we usually give it credit.

Values

Now that we know how to assign stuff with unprecedented sophistication, let's talk about what actually happens when we do so.

As it turns out, it's surprisingly simple — I don't know why it was never explained to me in simple terms. There was always talk about closures, bound and free variables, passing by value vs. passing by references, and confusing comparisons to Java and C.

In fact, none of this is relevant to understanding how Python assignment works — as we shall see, it all boils down to a hierarchy of dictionaries, with a pretty straightforward logic of how to get, set and delete their items.

To understand this, we first have to understand Python values; and here, the simple explanation is that they are just objects, floating in space. That's obviously true for class instances — when we define a class and instantiate it, we get the quintessential object:

>>> class A:
...     pass
>>> a = A()
>>> a
<A object at 0x...>

But even native types, like numbers and strings, are objects — they have their own attributes and methods. For example, a string has an upper method, which returns the same string in uppercase:

>>> 'hello'.upper()
'HELLO'

And an integer has a bit_length method, which returns how many bits it takes to encode it. It's a bit trickier to write, since the dot after the digit is parsed like the fraction of a float, not an attribute:

>>> 5.bit_length()
SyntaxError: invalid decimal literal

But it works if we just add a space, or wrap the number in parentheses:

>>> 5 .bit_length()
3
>>> (5).bit_length()
3

In fact, even functions are objects. We can print them out, and they can have attributes:

>>> def f():
...     pass
>>> f
<function f() at 0x...>
>>> f.x = 1
>>> f.x
1

And the same holds true for classes — not their instances, mind you, but the class itself. It's printable, and supports attributes just as well:

>>> A
<class A>
>>> A.x = 1
>>> A.x
1

This has really interesting ramifications, which we'll address when we talk about functions and classes. For now, the important point is that in Python, they are all objects, floating in space. Python allocates them for us, manages their internal representations, and frees up their resources after we're done — for us, they're just there.

Names

Still, we need a way to refer to these objects, which is where names come in. Names are simple strings, or keys, that point to a particular object.

For those of you coming from other languages, the bottom line is that Python doesn't pass anything by value, only by reference — because assignment doesn't deal with values at all, only with references.

To understand what this means in practice, suppose we have a list of three numbers, x, and we assign it to y — what happens if we then modify y?

>>> x = [1, 2, 3]
>>> y = x
>>> y.append(4)

When you see y = x, don't think about assignment operators or copy constructors; Python just binds a new name, y, to the object referenced by the old name, x. If we then append 4 to y, we're actually appending 4 to the list object, whatever its name — which is why it's "also" there when we call that object by x:

>>> x
[1, 2, 3, 4]

Similarly, if we have a function that appends a number to a list, the value is never copied into the function:

>>> def append5(items):
...     items.append(5)
>>> append5(x)

When you see append5(x), don't think about move semantics or copy elision; Python simply invokes a function that binds the object referenced by x to its parameter name, items. 5 is then appended to items, which is, once again, the same list object:

>>> x
[1, 2, 3, 4, 5]

A more romantic way to remember this is by Shakespeare's quote: "a rose by any other name would smell as sweet". In our case, a value by any other name would still be the same object, just floating in space.

Scopes

So, if names are just a way to reference values, what's left to understand is how Python manages the mapping between the two — and the most obvious way to do it would be with dictionaries.

Here's how it works: every unit of code is associated with such a dictionary, which is called a scope. Assignment is basically syntactic sugar for scope[name] = value, and name resolution is little more than scope[name].

In fact, these scopes are available as literal dictionaries using the builtin functions locals and globals. When we start a Python interpreter, we land in the first, outermost, or so called global scope. Whenever we indent into a function or a class, we create a new scope, associated with that unit of code. Consider this:

>>> x = 1
>>> def f():
...     y = 2
...     def g():
...         z = 3
...         print(locals())
...         print(globals())
...     g()

We have the name x bound to the value 1 in the global scope; then inside the function f, we have y bound to 2; and further inside g, we have z bound to 3.

If we print the locals and globals from there, then we can call f, which will call g, which will let us glimpse how those scopes look from inside it:

>>> f()
{'z': 3}
{..., 'f': <function f() at 0x...>, 'x': 1}

And indeed, we see z in the last, innermost, or so called local scope; and we see x in the global scope, among some other standard global variables, like f itself.

If this still hasn't sunk, here's one last example to really drive the point home. If we grab the global dictionary, we can literally treat it as such:

>>> d = globals()
>>> d['x'] = 1
>>> x
1
>>> del d['x']
>>> x
NameError: name 'x' is not defined

Any key we assign in the dictionary becomes available to us as a global name, as a "variable"; and if we delete the key, that name, or "variable", disappears as well.

That's why it's a bad idea to compare Python names to C or Java variables, or Python scopes to something like the program's heap or stack — it's a different beast entirely, resolved dynamically, almost like a dictionary.

That "almost" part is important, thought. There are some nuances, and to understand them fully, let's break the process down, and talk separately about name resolution, where we get a value; assignment, where we set a value; and deletion, where we remove a value.

Name Resolution

First, we have name resolution. Consider the following code, which is pretty similar to what we had before:

>>> x = 1
>>> def f():
...     y = 2
...     def g():
...         z = 3
...         print(z)
...         print(y)
...         print(x)
...         print(w)
...     g()

What happens when we call f, which calls g? It attempts to print z, but in order to do so, it has to get the value associated with the name — to pluck the object out of the space it float in. Luckily, it doesn't have to go very far: z is bound to 3 in g's local scope, which is, not surprisingly, the first place that Python checks.

Then, it attempts to print y, which is not in g's local scope. However, g is defined inside f, which has y bound to 2 in its local scope; and since this is the next scope that Python checks, the name is resolved soon enough.

Then, it attempts to print x, which is in neither g's nor f's local scopes, so we have to go farther. And indeed, x is bound to 1 in their module's scope — that's the global scope, whether we're talking about a file or an interpreter session.

Only w, which is missing from all the scopes, can't be resolved; so Python raises a NameError to complain about it:

>>> f()
3
2
1
NameError: name 'w' is not defined

And by the way, Python has some optimizations in place that determine which names come from what scope in advance, as soon as the function is defined and its code is compiled — so the process is not quite so naïve. Still, I find that thinking about it as a progressive hierarchy of dictionaries helps build an intuition about how scopes behave, regardless of the implementation details.

Assignment and Deletion

Next, we have assignment — finally, right? — and it's even simpler, because it always operates only on the local scope. Suppose we have:

>>> x = 1
>>> def f():
...     x = 2
...     print(x)
>>> f()
2
>>> x
1

When we print x inside f, it resolves to 2; but once we leave f, it resolves to 1 again. Hopefully, this doesn't surprise you; but ask yourself why this happens — after all, local variables are not stored in a stack.

Here's why: the name x is bound to the value 1 in the global scope, but inside f, the same name is bound to the value 2. So when f looks for x, it finds it in its local scope; while outside it remains unchanged. If we want, we can even access it from f through the globals dictionary, and see that there it's still 1:

>>> x = 1
>>> def f():
...     x = 2
...     print(globals()['x'])
>>> f()
1

In other words, assignment doesn't perform the same lookup as name resolution does. It doesn't reassign the global, original x; it just operates on the local scope straight away, even if this means obscuring, or shadowing, some names in the enclosing scopes.

Similarly, deletion always operates on the local scope; so if we try to do something like define x outside a function and delete x inside it, we're going to get an error:

>>> x = 1
>>> def f():
...     del x
>>> f()
UnboundLocalError: local variable 'x' referenced before assignment

At first sight, it looks a bit strange — local variable x referenced before assignment?

But now that we understand scopes, it should make some sense. Since Python treats deletion as an inherently local operation, it gets upset when we reference x — that is, when we try to delete it — before it was ever properly assigned in that local scope; which, you have to admit, it wasn't.

Selecting Scopes

Still, there are some ways to set and delete names in non-local scopes, like the global keyword. It's generally frowned upon, but occassionally useful; and anyway, it fits well with our thinking of scopes as a hierarchy of dictionaries, so lets take it for a spin.

If we add global x at the beginning of the code block, this keyword will effectively instruct Python that in this here function, the name x should be resolved in, assigned to and deleted from — not the local scope as usual, but the global one:

>>> x = 1
>>> def f():
...     global x
...     x = 2
>>> f()
>>> x
2

Notice that this still means that assignment only ever operates on one scope; it's just that it is rewired to operate on the outermost one, instead of defaulting to the innermost one.

Similarly, if we add global x before our deletion, Python simply makes a mental note that for the name x, it should consider the global scope. As a result, the deletion works; after calling the function, the name is no longer available:

>>> x = 1
>>> def f():
...     global x
...     del x
>>> f()
>>> x
NameError: name 'x' is not defined

But what about names that are stuck in the middle, in a limbo that is neither local nor global? Suppose we have f that sets y to 2, and then g inside it, which wants to set y to 3 or to delete it:

>>> def f():
...     y = 2
...     def g():
...         ?

If we go about it the usual way, g will default to its local scope, and either shadow the original y, or raise an UnboundLocalError like before. But if we declare global y, g will be wired to that scope, and skip f's intermediate one, where y is actually defined.

That's why, in Python 3, they added the nonlocal keyword. What it does is figure out the closest, innermost scope (except the local one) where some name is defined, and rewire name resolution, assignment and deletion to get, set and delete it from there.

To wit, if we add nonlocal y before we set it to 3, the value in f is actually reassigned, and y stays 3 even after g is over:

>>> def f():
...     y = 2
...     def g():
...         nonlocal y
...         y = 3
...     g()
...     print(y)
>>> f()
3

And if we delete y, it actually disappears from f's scope, and we get a NameError when we try to print it:

>>> def f():
...     y = 2
...     def g():
...         nonlocal y
...         del y
...     g()
...     print(y)
>>> f()
UnboundLocalError: local variable 'y' referenced before assignment

Late Binding

Lastly, I'd like to tell you about a strange bug, which stems from basic confusion about how scopes work — and that hopefully, we can resolve quite easily with our newfound understanding.

Suppose we have a list of functions, and we iterate with i from 0 to 9, each time defining a new function that returns the current i and adding it to the list:

>>> fs = []
>>> for i in range(10):
...     def f():
...         return i
...     fs.append(f)

When we call the first function in the list, we expect to get 0:

>>> fs[0]()
9

But we get 9.

In fact, we can call any function in the list, and we'd get 9 every time:

>>> fs[5]()
9

And it gets weirder — if we set i to 10, all of the functions seem to change to reflect this:

>>> i = 10
>>> fs[0]()
10
>>> fs[5]()
10

Suddenly, they all return 10.

But this is only weird until we realize that in fact, we haven't created ten different functions — we have created ten identical ones, all of which return not a number, mind you, but the name i. Since i is not defined in their local scope, name resolution proceeds as you might expect, and finds it in their global scope — which is the same for all of them.

After the loop ends, the name i is bound to the value of its last iteration, which is why all the functions return the same 9; and if we explicitly set it to 10, it seems to change them, when in fact they're doing exactly the same thing as before — essentially, return globals()['i'], which now happens to be 10.

This "bug", or rather behavior, is called late binding, because the values are bound later than we might expect.

To solve it, we need to introduce an intermediary scope, in which the specific value of i would be retained. Suppose we define create_f, which receives an i and returns a function accordingly:

>>> def create_f(i):
...     def f():
...         return i
...     return f

If we call it ten times:

>>> fs = []
>>> for i in range(10):
...     f = create_f(i)
...     fs.append(f)

We still end up with ten identital functions that return a non-local i, right? But this time, they're not all defined in the same scope and with the same i.

The first one, for example, is defined in the scope of create_f(0), where i is bound to 0. If we were to de-parameterize the function by fixing its argument to this value, we'd get:

>>> def create_f0():
...     i = 0
...     def f():
...         return i
...     return f

So when f needs to resolve i and goes looking for it, that's what it finds:

>>> fs[0]()
0

A different f behaves the same way, but is defined in a different scope — like create_f(5), where i is bound to 5, as if in:

>>> def create_f5():
...     i = 5
...     def f():
...         return i
...     return f

So that's what it finds:

>>> fs[5]()
5

And so on. In each iteration, while we still have the relevant value of i, we create a new scope by invoking create_f, and in that particular intermediary scope, i has the correct value — the extra scope effectively fixes it via its parameter. Any function defined inside it will look through this scope when resolving names; so every function has a different lookup, and arrives at a different result.

Scope Injection

Another way to address late binding would be to inject i directly into the local scope of each function, instead of wrapping them in intermediary ones. At first glance, it's not clear how to do that; if we just add a local variable, like x, and return this instead of i:

>>> fs = []
>>> for i in range(10):
...     def f():
...         x = i
...         return x
...     fs.append(f)

We still end up with the same lookup; we just shifted the problem to another name:

>>> fs[0]()
9

But as we've seen, a function's scope contains not only its local variables, but also its parameters — and those can have default values. So if we define each function to have a parameter with its default value set to i:

>>> fs = []
>>> for i in range(10):
...     def f(x=i):
...         return x
...     fs.append(f)

It actually works:

>>> fs[0]()
0
>>> fs[5]()
5

Although it's somewhat less elegant, and has the strange side effect that now our functions have... well, a parameter, which can effectively override their result:

>>> fs[0](2) # ???
2

But hey, it's a quick and dirty way to fix the problem (assuming we don't abuse the parameter, and let it default to its "natural" value) without introducing intermediary scopes.

The reason it works is that while a function's body isn't executed until it's called — by which time, i is already 9 for all the functions — here, i is used as part of the function's definition, with a different i in each iteration.

This means each specific i is retained; this time not inside a particular scope, but as part of each function's object. We can even find it in the function's __defaults__ attribute:

>>> fs[0].__defaults__
(0,)
>>> fs[5].__defaults__
(5,)

But we'll cover that when we talk about functions at depth.

For now, the point is that when the functions are called, they're still identical — they all simply return a local variable, x — but this x is provided with a different default value every time. Before, they all returned the non-local variable i, but it was resolved in a different scope. So in both cases, we were trying to secure proper name resolution — first by wiring it through an additional, dedicated scope, and then by hacking our way into the storage of its existing one.

Anyway, late binding is not that common, but it can cause real enough bugs; and what's more, it thoroughly shakes our understanding of scopes, which is a good way to make sure that we have a good grip on the key concepts and the way they interact.

Assignment: Conclusion

So yeah, assignment.

As you can see, it's a simple topic, but there's a lot to say about it — and I've deliberately left out the walrus operator, which is a whole story unto itself.

To sum up, we saw that Python values are objects, floating in space, and referred to by names, which are stored in a hierarchy of dictionaries. We saw how name resolution traverses this hierarchy, from the local scope to the global one — while assignment and deletion work on a single scope; usually the local one, but this can be rewired with the global and nonlocal keywords.

Understanding this, together with mastering the syntax to effectively destructure data, will make your life as a Python developer much easier; and it will come in handy in the future, when we talk about functional programming, attribute resolution, and other more advanced topics.

Assignment: Exercises

• Assign the values 1-5 from numbers to the names a-e in a single line, where:
• numbers = [1, 2, 3, 4, 5]
• numbers = [[1], [2], [3], [4], [5]]
• numbers = [1, [2, [3, [4, [5]]]]]
• numbers = [1, 0, [2, [3], 4, 5, 0, 0]]
• numbers = {'a': [1], 'b': [0, 2], 'c': [0, 0, 3], 'd': [0, 0, 0, 4], 'e': [0, 0, 0, 0, 5]}
• Given:

def f():
    return 1, 2

def g():
    return 3, 4, 5

def h():
    return 6, 7, [8, 9]

• Assign the values returned from f and g to the names a-e in a single line.
• Assign the last value returned from g and the values returned from h to the names a-e in a single line.
• Replace what with the appropriate assignment to make the loops work.

def q1():
    numbers = [[1, 2, 3], [1, 2, 3], [1, 2, 3]]
    for what in numbers:
        assert x == 1
        assert y == 2
        assert z == 3

def q1():
    numbers = [[1, 2, 3], [1, 2, 3], [1, 2, 3]]
    for what in numbers:
        assert x == 1
        assert y == 2
        assert z == 3

def q1():
    numbers = [[1, 2, 3], [1, 2, 3], [1, 2, 3]]
    for what in numbers:
        assert x == 1
        assert y == 2
        assert z == 3

def q1():
    numbers = [[1, 2, 3], [1, 2, 3], [1, 2, 3]]
    for what in numbers:
        assert x == 1
        assert y == 2
        assert z == 3

def q1():
    numbers = [[1, 2, 3], [1, 2, 3], [1, 2, 3]]
    for what in numbers:
        assert x == 1
        assert y == 2
        assert z == 3

Assignment: Further Reading

• A good talk about Python scopes and how they work is , by Ned Batchelder, from 2015.
• Python's documentation is particularly sparse when it comes to assignment; you can glimpse pieces in discussions about , or reconstruct its logic from its , but other than that I haven't found anything useful.
• , from javascript.info, elaborates on data destructuring in Javascript.
• discusses extended iterable unpacking, which we called star capturing.
• introduces additional unpacking generalizations, of which we discussed star exploding.
• discusses the way in which Python names are resolved in statically nested scopes; what we called a hierarchy of dictionaries.
• introduces access to names in outer scopes with the nonlocal keyword.
• introduces the builtin set type; in case you're not very familiar with it, it's actually quite handy — and you can find out more about it in .