Networking: How the Internet Works

From One Process to Many

So far, we've been talking about a single program: its names, functions, objects and modules, and in the last chapter, how fast it runs. But programs don't live in a vacuum — they almost never exist on their own. So the interesting question becomes, how do many programs, or processes, work together?

If these processes run on the same machine, we call it inter-process communication, or IPC: one process might write a file, which another one reads; or, they might share a specially mapped region of memory. They can also use a pipe, or a Unix domain socket — a two-way communication channel provided to them by the operating system:

>>> import socket
>>> a, b = socket.socketpair()
>>> a.family
<AddressFamily.AF_UNIX: 1>
>>> a.sendall(b'hello')
>>> b.recv(1024)
b'hello'
>>> b.sendall(b'hi there')
>>> a.recv(1024)
b'hi there'

Here, socketpair gives us a connected pair of Unix domain sockets (that's the AF_UNIX family), so that anything we write into one end comes out the other — in both directions. Of course, in a real scenario, each end would belong to a different process, like a parent and the child it forked — but the principle is the same: it's a way for two processes to communicate, to exchange bytes, on the same machine.

But synchronizing processes on the same machine is not actually all that interesting. What we really want is processes on different machines to work together — whether it's in the same building, or on the other side of the world. This is the internet; and the plan is to build it from zero, in what's known as a stack of abstractions — a series of layers, where each one solves a smaller problem, so that the next one can forget about it, and build on top of its solution to address a bigger problem.

First, we'll have to connect two machines with a cable; then, figure out a way to connect many machines in a shared medium. We'll need some addresses and routers to connect different networks into one network of networks; and then some ports, to connect not just machines, but processes. Finally, we'll arrive at the so-called application protocols, like DNS and HTTP — the languages processes actually speak.

This is going to be a bit of a crash course, not a networking course. Each of these layers can (and does) have a course of its own, and we won't go into nearly as much detail as any of them. I just want to make sure that we're speaking the same language, and to give you an idea of why things are the way they are — because my main point in this chapter is that the internet is not magic. A lot of people seem to think it is: something that just works, something that just is. But if you actually look under the hood, it's pretty underwhelming — and I mean that in the best possible way. At the end of the day, it's just a bunch of processes running on a bunch of servers, connected by a bunch of cables. The process that runs Google, for example, is no different: it's running on some server, has opened a socket on some port, and is accepting connections, receiving a piece of text called a request, running some code and writing back a response. That's it; that's all of the internet.

And what's more, everything that happens between these processes is just a bunch of bytes: zeroes and ones, represented by different voltages on a copper wire, pulses of light in an optical fiber, or radio waves in the air. We're going to spend a good part of this chapter in Python, disassembling real recorded traffic with the struct module, byte by byte, and seeing that there's really nothing secret about it. If anything, Python makes it too easy: you just import socket, and then connect and send. You never care about the fact that somewhere, behind the scenes, a port is allocated, a route is chosen, and a packet is handed to your default gateway; the operating system takes care of it. So in this chapter, we'll be talking about the rest; and in the next one, we'll get back to the socket module itself.

Layers, Headers and Parcels

Before we talk about the different layers, let's understand the central idea behind this approach. In a network, both the software and the hardware are organized in layers; and each layer has a protocol, through which it communicates with its counterpart — that is, the same layer — on the other machine. So layer n on my machine talks to layer n on yours; and to do so, it uses layer n-1 to actually carry its message there, as-is, and lets layer n+1 ride on top of it.

To understand this, think of a phone call: I have an idea, which my brain turns into words; my vocal system then turns those words into sounds; and the phone turns those sounds into electrical signals, which are then sent over the wire to you. On your side, the phone translates those electrical signals into sounds; which your ears translate into words; which your brain translates into an idea.

So logically, ideas are talking to ideas, and words are talking to words; but physically, the entire process happens top-down on my side, crosses over to yours at the very bottom, and propagates back up — in the shape of a big U.

Every message in every protocol has two parts: metadata, which describes the message, and data, which contains its payload. The metadata is usually placed in a header, and sometimes in a footer or trailer; it describes where the message is going, where it's coming from, how long it is, and other information that helps handle it. The data, then, is an opaque blob: something that the layer doesn't care about, but is simply charged with delivering. And this opacity is exactly what lets this data contain another protocol's header and data, and so on, in a recursive fashion.

Or, think of a parcel being passed around. I wrap my message and hand it over; each handler peels off their own layer, reads what's relevant to them, and passes the inside on — where it makes sense to someone else. On the way back, each one wraps it up again.

The standard names for these layers come from the OSI model, a standard that was developed by the ISO organization in the late 1970s. It defines seven layers: physical, data link, network, transport, session, presentation and application. The standard itself wasn't a great technological success, but its terminology stuck; and since the designers of the internet stopped at layer 4, lumping everything above it in a single application layer, we're left with five: the physical layer, the link layer, the network layer, or IP, the transport layer, which includes TCP and UDP, and the application layer.

To work with headers in Python, we first have to figure out how to convert numbers to bytes and back; we've already seen int.to_bytes and the endianness problem in the native types chapter, and the struct module does the same for entire records described by a format string:

>>> import struct
>>> struct.pack('<I', 1)
b'\x01\x00\x00\x00'
>>> struct.pack('>I', 1)
b'\x00\x00\x00\x01'
>>> struct.pack('!HI', 80, 1)
b'\x00P\x00\x00\x00\x01'
>>> struct.calcsize('!HI')
6
>>> struct.unpack('<HH', b'\x01\x00\x00\x01')
(1, 256)

Let's go over this again: < means little endian and > means big endian — the same Gulliver's dilemma we encountered before; and ! means network byte order, which is big endian, and is the convention used by internet protocol headers. I is an unsigned 4-byte integer and H is an unsigned 2-byte integer; calcsize tells you how many bytes this format takes; and unpack always returns a tuple, even if there's only one value. Also, notice how 80, packed into two bytes, actually looks like b'\x00P', because 80 happens to be the ASCII code for P.

Now let's take a look at a toy packet, from an exercise in my Introduction to Information Security course. Suppose it has three imaginary layers: layer 1 has a sender ID, a receiver ID and the content size, all 4 bytes each, followed by the content itself; layer 2 has a 4-byte session ID, followed by the content, followed by a 4-byte checksum, as a footer; and layer 3 has a 4-byte message ID, followed by the message. All the numbers are represented as unsigned integers in little endian:

>>> packet = b'\x08\x00\x00\x00\xf6\x01\x00\x00$\x00\x00\x00\x03\x00\x00\x00\x0c\x00\x00\x00I think, therefore I am.\xca\xcd\x00\x00'
>>> sender, receiver, size = struct.unpack('<III', packet[:12])
>>> sender, receiver, size
(8, 502, 36)
>>> layer2 = packet[12:12 + size]
>>> session, = struct.unpack('<I', layer2[:4])
>>> checksum, = struct.unpack('<I', layer2[-4:])
>>> session, checksum
(3, 52682)
>>> layer3 = layer2[4:-4]
>>> message_id, = struct.unpack('<I', layer3[:4])
>>> message_id, layer3[4:].decode()
(12, 'I think, therefore I am.')

What we do is unpack layer 1's header, which is a sender ID (8), a receiver ID (502) and the content size (36), then slice the content accordingly. In it, we see that the first 4 bytes are layer 2's session ID, and the last 4 bytes are its checksum, which means everything in the middle is layer 3. Here, we see the first 4 bytes are the message ID (12), and the rest is the message itself: "I think, therefore I am." — which we can decode. We don't even bother verifying the checksum, since we don't know how exactly it was computed, and as layer 3 we don't really need to.

Notice how in each layer, we were only interested in the metadata of that layer, treating the rest as an opaque blob. This is exactly how real network traffic looks — except it has more fields, which we'll peel off similarly in the next sections, with a real recording.

Shouting on the Bus

Let's start with two machines. We can connect them with a cable; but it won't work just by plugging a wire in — each one needs to have a network interface card, or a NIC, a piece of hardware that knows how to signal bits on the wire. It also needs a driver, some software that speaks a certain protocol over this medium: old computers did exactly that, using protocols like SLIP over a serial line; and when you pair your headphones with your phone over Bluetooth, it's still essentially a link between two parties.

When we have many machines, connecting all of them to all of them with cables would require $n$ machines to be connected with $\frac{n(n-1)}{2}$ cables:

>>> for n in (2, 5, 100, 1_000_000):
...     print(n, n * (n - 1) // 2)
2 1
5 10
100 4950
1000000 499999500000

Five machines would need ten cables, a hundred would need 4,950, and a million would require about half a trillion. Not very practical, is it? Especially when one of these cables needs to cross the Atlantic ocean, so you can send an email to a server in America. So a smarter solution is a shared medium, or bus: everyone connects to it, and then everyone can shout their messages into it, and the medium broadcasts it to everyone, so everyone can hear everyone. In WiFi, the bus is literally a radio frequency: all your devices are shouting into it all the time, "hey, get me Google", "hey, get me YouTube". With Ethernet, it's cables plugged into a hub or a switch: this is called a local area network, or a LAN.

Now, since we're shouting into the wind, we have to address our messages to someone: every NIC has a MAC address, or media access control address, a unique identifier of 48 bits, usually written as six bytes of two hex characters each, separated by colons. Every frame contains a destination MAC address and a source MAC address: every NIC compares the destination address to its own, and ignores the message if it's not for it — unless it's a broadcast, or a multicast, in which case everyone receives it. You can also configure the NIC to work in promiscuous mode, in which case it forwards everything to the system, and that's how network sniffers like Wireshark work, capturing and displaying everyone's traffic on a shared network.

Finally, when we're shouting into the wind, we have to be prepared for someone else to be shouting at the same time, in which case we'll have a collision. With classic Ethernet, when two machines transmit at the same time, they both notice this clash and stop; but then, they wait for a random amount of time and try again. This simple idea, called exponential backoff, we'll see again when we talk about message buses later.

A recording like the ones Wireshark makes is called a pcap file, and I have such a file with a few seconds of traffic from a small home network, used in an online information security course. Its format is quite simple: first, there's a 24-byte global header, and then, for every captured frame, there's a 16-byte record header — with the timestamp in seconds and microseconds, and the captured and original lengths of the frame — followed by the actual bytes of the frame:

>>> data = open('home_anon.pcap', 'rb').read()
>>> len(data)
1443
>>> magic, major, minor, zone, sigfigs, snaplen, linktype = struct.unpack('<IHHiIII', data[:24])
>>> hex(magic), major, minor, linktype
('0xa1b2c3d4', 2, 4, 1)
>>> def read_pcap(data):
...     offset = 24
...     while offset < len(data):
...         seconds, micros, size, _ = struct.unpack('<IIII', data[offset:offset + 16])
...         offset += 16
...         yield seconds + micros / 1e6, data[offset:offset + size]
...         offset += size
>>> frames = [frame for timestamp, frame in read_pcap(data)]
>>> len(frames)
7

If we decipher it correctly, the magic number 0xa1b2c3d4 indicates that this is indeed a pcap file, and the fact that we got it right in little endian format means it was saved in the same endianness. Next, we see that the version is 2.4, and the link type is 1, indicating Ethernet; and then, read_pcap is a generator, like the ones we've seen in the generators chapter, yielding the timestamp and the raw bytes of each frame — there are seven of them. Decoding their Ethernet header:

>>> for frame in frames:
...     destination, source, ethertype = struct.unpack('!6s6sH', frame[:14])
...     print(source.hex(':'), '->', destination.hex(':'), hex(ethertype))
02:00:00:00:00:03 -> 01:00:5e:00:00:fb 0x800
02:00:00:00:00:02 -> 01:00:5e:00:00:fb 0x800
02:00:00:00:00:0c -> 01:00:5e:00:00:fb 0x800
02:00:00:00:00:07 -> 02:00:00:00:00:8a 0x800
02:00:00:00:00:8a -> 02:00:00:00:00:07 0x800
02:00:00:00:00:07 -> 02:00:00:00:00:8a 0x800
02:00:00:00:00:8a -> 02:00:00:00:00:07 0x800

What I'd like you to notice is that the first three frames are all sent to 01:00:5e:00:00:fb, a multicast address, by three different devices announcing themselves to the whole network; the last four frames are a conversation between two devices, 02:00:00:00:00:07 and 02:00:00:00:00:8a. (I actually changed the MAC addresses in this copy of the recording to make it a bit less identifiable; a first byte of 02 indicates a locally administered address, as opposed to the one burned into the card by its manufacturer, whose ID would normally be in the first three bytes.) What's more, notice that their EtherType is 0x800 — it's an indication to the next layer up, the network layer, that the payload is an IPv4 packet.

As an aside, how does a machine even know what MAC address belongs to a given IP address? A protocol called ARP, the address resolution protocol, sometimes called layer two and a half, is used for that: a host asks the entire LAN "hey, who has this IP address", and the owner answers, "it's me; this is my MAC address". Hosts also listen to everyone else's answers, and update their own tables accordingly, to save time; but this is also what makes ARP poisoning possible, when an attacker announces "it's actually me, I'm the router", and gets in the middle of your traffic without you knowing.

Addresses, Gateways and NAT

Now, how do we connect different LANs to each other? We add another layer of abstraction: just like every machine has a MAC address, which is kind of like an ID card for that medium — now, every machine also gets an IP address, kind of like an ID card for this new, bigger layer. This is a number that says "it's me, and I'm part of the world". Connecting all these different networks together is the job of internet service providers, or ISPs: so if I ask my ISP, let's say Bezeq in Israel, for Google — and Google happens to be sitting in America, under AT&T — then Bezeq needs to talk to AT&T, get Google, and pass it on to me. In between them is some infrastructure that most people never deal with directly: the backbone of the internet. That box in your home that's called a modem is actually the box that connects you to your ISP — that's how they can charge you money, after all.

Let's take a closer look at this new ID card. For that, we can use the ipaddress module:

>>> import ipaddress
>>> ip = ipaddress.ip_address('10.0.0.7')
>>> ip.packed
b'\n\x00\x00\x07'
>>> int(ip)
167772167
>>> ip.is_private, ipaddress.ip_address('8.8.8.8').is_private
(True, False)
>>> 2 ** 32
4294967296
>>> ipaddress.ip_address('127.0.0.1').is_loopback
True
>>> ipaddress.ip_address('2a04:4e42::223').packed.hex()
'2a044e42000000000000000000000223'

So, an IP address is actually just 32 bits, which we write as four bytes, separated by dots. The packed attribute gives us the four bytes, and if you squint, you can see that the 10 is actually \n in ASCII, again. And int gives us the number, and 2 to the power of 32 is indeed 4.3 billion — which turned out to be too few for all the devices that wanted to get online. That's why we now have IPv6, which uses 128 bits (so the python.org address you see there is actually 16 bytes). But despite that, its adoption is still pretty slow, because changing stuff is hard.

What did become popular is a different solution: it's called network address translation, or NAT. You can think about it like this: imagine you have several groups of people, where instead of everyone talking to everyone, each group has a representative — and this representative goes out into the world on behalf of their group, talks to whoever's necessary, and comes back with an answer for each. This way, only the representative needs an ID card, a public IP address; and the rest of the group can be identified by some private address. At home, the representative is the gateway — usually called the router — and it's usually the same box as the modem that was provided to you by your ISP. Inside your house, though, all the different machines have their own, private IP addresses, allocated from special ranges reserved for this purpose: 10.x.x.x, 172.16.x.x to 172.31.x.x, and 192.168.x.x. It's usually the router that allocates them, using a protocol called DHCP; so your default gateway at home might be 10.0.0.138, or 192.168.0.1. This way, when my laptop, sitting at 10.0.0.7, wants to get Google, it asks its router — which then goes out into the real internet, with its public address, and when an answer comes back, it knows to look for my laptop in the network it spans, and pass the answer along.

But how does a machine know whether it should talk to someone directly, or go through its gateway? This is where subnets come in. The network that this machine is in is described by a prefix: for example, 10.0.0.0/24 means that the first 24 bits are fixed, and the last 8 can vary — so it can have 256 different addresses, and my laptop is one of them. We can similarly describe a network with 16 bits left — so, 65,536 different addresses:

>>> network = ipaddress.ip_network('10.0.0.0/24')
>>> network.netmask, network.num_addresses
(IPv4Address('255.255.255.0'), 256)
>>> ip in network, ipaddress.ip_address('8.8.8.8') in network
(True, False)
>>> int(ip) & int(network.netmask) == int(network.network_address)
True
>>> ipaddress.ip_network('10.0.0.0/16').num_addresses
65536

So the mask is 255.255.255.0, and an address is inside this subnet exactly if, when we AND it with this mask, we get the network's address; and that's exactly what the in operator does for us. If an address is inside our subnet, it means we're supposed to talk to it directly, on this LAN — so, by its MAC address. If it's outside the subnet, it means we have to pass the packet to our gateway. There are two special cases, though: one is 127.0.0.1, called localhost, which means "me" — so it never leaves your machine, which is why we'll use it in the next chapter when we want to run both a server and a client on the same box. The other one is 0.0.0.0, which means "any address".

OK, so let's go back to our recording, and look at the sixth frame. This one's payload is an IPv4 packet, and its header is 20 bytes of some fixed fields:

>>> header = frames[5][14:34]
>>> (version_ihl, tos, length, ident, fragment,
...  ttl, protocol, checksum, source, destination) = struct.unpack('!BBHHHBBH4s4s', header)
>>> version_ihl >> 4, (version_ihl & 0xf) * 4
(4, 20)
>>> length, ttl, protocol
(62, 64, 17)
>>> ipaddress.ip_address(source), ipaddress.ip_address(destination)
(IPv4Address('10.0.0.7'), IPv4Address('8.8.8.8'))
>>> sum(struct.unpack('!10H', header)) % 0xffff
0

The first byte is actually two fields: the version, which is 4, and the header length in 4-byte words, which is 5 (so, 20 bytes). The total length is 62 bytes, and the TTL is 64. The protocol is 17, which means UDP. The source is my laptop, 10.0.0.7, and the destination is 8.8.8.8, which happens to be Google's public DNS server. As for the checksum: we sum all the 16-bit words in the header together, and then do some funky math on them, called ones' complement arithmetic; the result should be 0xffff if the header is intact — and that's why the sum should be 0, modulo 0xffff.

Also notice the layering at work: on the IP level, this packet is destined for Google; but if you look at its Ethernet header, it's actually destined for my router's MAC address, because the link layer can only deal with its next hop. It's only on the IP level that this packet can have a notion of the world, and know where it's ultimately going to end up.

Routing: Getting There

Of course, even if everyone gets an address, we still need a path; a packet needs to get from my laptop in Tel Aviv to some server in California. That's the entire point of IP, really; much like the postal system, its main purpose is to deliver packets from senders to receivers, router by router, where each router only needs to decide on the next hop, based solely on the destination address.

And just like the post office, it's a best-effort delivery: if there's a lot of traffic, packets can get dropped; or they can arrive out of order, or even several times — IP doesn't promise anything more. If you need such guarantees, it's someone else's problem, as we'll see when we get to TCP.

The decision of the next hop, then, is a lookup into a table: a list of networks, expressed as prefixes, and where to forward packets going to each one of them. When there are several options, the most specific one is chosen; this is known as longest-prefix matching. In fact, your laptop has the simplest version of such a table: everything in my subnet, it says, go directly there; and everything else, 0.0.0.0/0, go to the default gateway. To give you a taste of how this works, here's how such a table might look for a toy example:

>>> table = {
...     ipaddress.ip_network('10.0.0.0/24'): 'deliver directly on the LAN',
...     ipaddress.ip_network('8.8.8.0/24'): 'hand to the Google peering link',
...     ipaddress.ip_network('8.0.0.0/8'): 'hand to the transit provider',
...     ipaddress.ip_network('0.0.0.0/0'): 'hand to the default gateway',
... }
>>> def route(address):
...     address = ipaddress.ip_address(address)
...     matches = [network for network in table if address in network]
...     return table[max(matches, key=lambda network: network.prefixlen)]
>>> route('10.0.0.12')
'deliver directly on the LAN'
>>> route('8.8.8.8')
'hand to the Google peering link'
>>> route('8.8.4.4')
'hand to the transit provider'
>>> route('151.101.0.223')
'hand to the default gateway'

And route simply collects all the networks that contain this address, and selects the one with the largest prefixlen. 10.0.0.12, for example, is delivered directly; 8.8.8.8 matches both the /8 prefix and the more specific /24 prefix, so it's routed to the latter; 8.8.4.4 matches the /8 only; and python.org's address, 151.101.0.223, doesn't match any of these, and will be sent to the default gateway, 0.0.0.0/0, which matches everything. Real routers have hundreds of thousands of such entries, and learn them from each other: between ISPs, there's a protocol called BGP, which has its own vulnerabilities, and whose misconfigurations occasionally make big chunks of the internet disappear for a while.

If there was no agreement between all these tables, packets could end up in a loop, which is why there's another important field, called the TTL, or time to live. It's decremented by every router as the packet passes through, and if it ever reaches zero, the packet is dropped, and the sender is usually notified with an ICMP message. That's what the traceroute command is based on: it sends packets with a TTL of 1, 2, 3 and so on, and lists all the routers that "answer" with an ICMP message, until it gets to its destination. In fact, if we look at the TTLs of our DNS query and its reply:

>>> frames[5][14 + 8], frames[6][14 + 8]
(64, 120)

We can see that my laptop sent its request with a TTL of 64, and that the reply arrived with 120. The initial value isn't actually part of the packet — but systems usually start with nice, round numbers like 64, 128 or 255, so it's likely that this reply started with 128, and crossed about eight routers on its way back from Google.

As an aside, this is also why CDNs exist. The fewer hops, the faster the packet — so if you're browsing the web from Israel, there's no need to cross the ocean to get to the content you're looking at, if it can be cached somewhere close to you. In fact, we'll see python.org served from just such a cache later on in this chapter.

Ports, and the Simplest Transport

Having connected all the computers in the world, we need to connect all the programs in the world, and to do that, we need a finer resolution: ports. A computer runs multiple processes, so we need a way to specify which of them we want to talk to; and that's what ports are. They're just 16-bit numbers between 0 and 65535, which are used to indicate an endpoint within a machine.

The way it works is that servers — the programs that provide some service, like a webserver or a DNS server — listen for a connection on some fixed, well-known ports. Some of the famous ones are 80 for HTTP, 443 for HTTPS, 22 for SSH and 21 for FTP; DNS runs on 53, and we can actually look them up with the socket module:

>>> for service in ('http', 'https', 'ssh', 'ftp', 'domain'):
...     print(service, socket.getservbyname(service))
http 80
https 443
ssh 22
ftp 21
domain 53
>>> socket.getservbyport(5353)
'mdns'

In fact, these are defined in the system's services database — on Unix systems, it's /etc/services. Port 5353, for example, is used for mDNS — multicast DNS, which is what the three weird announcements we saw before actually were.

As for clients — the programs that use some service, like a web browser — they allocate themselves a temporary, high-numbered port, called an ephemeral port, and use that to initiate a connection. So when I use Chrome to browse the web, it might pick some port like 53716, and use it to connect to 443 of one of Google's servers — that is, talk to the program that provides the HTTPS service, which is probably running on a machine in one of their datacenters.

As an aside, this is the operating system's ephemeral port range — on my Mac, at least (Linux uses a different default range):

$ sysctl net.inet.ip.portrange.first net.inet.ip.portrange.last
net.inet.ip.portrange.first: 49152
net.inet.ip.portrange.last: 65535

If you check, you'll find that all the client ports I'll be using in this chapter and the next are within this range.

So, another type of multiplexing: just like a gateway hides a whole network behind one public address, a machine hides all its programs behind one address, and uses a port to distinguish between them. In fact, that's what NAT relies on as well: the router actually rewrites the source port, so that when a reply arrives, it can tell which of the internal machines it was addressed to.

The simplest protocol that does this is UDP, or user datagram protocol. It has a very simple header: a source and a destination port (both 16 bits), a length and a checksum (both 16 bits), followed by the data. That's it: no connection, no acknowledgment, no ordering, no retransmission: you send a datagram, and you hope.

Let's go back to our DNS query, and see what it looks like:

>>> segment = frames[5][34:]
>>> source_port, destination_port, length, checksum = struct.unpack('!HHHH', segment[:8])
>>> source_port, destination_port, length
(52126, 53, 42)
>>> len(segment)
42
>>> segment[8:]
b'0\x04\x01\x00\x00\x01\x00\x00\x00\x00\x00\x00\x03www\x08facebook\x03com\x00\x00\x01\x00\x01'

The query was sent from 52126, which is an ephemeral port, to 53, which is the DNS port; and it's 42 bytes long, accounting for the 8-byte header and 34 bytes of payload. If we look at the payload, we'll see www, facebook and com in it, which we'll decode when we talk about DNS.

I'd tell you a UDP joke, but you probably wouldn't get it.

Anyway, why would you use something so flimsy? Because it's cheap: for something like a DNS lookup, which is a small question followed by a small answer, establishing a connection would be more costly than the exchange itself, and if the answer doesn't arrive, we can always ask again. Video calls and games, too, prefer UDP: if a packet doesn't arrive on time, it's useless anyway.

TCP: Did You Get That?

And that's why UDP is great for some applications, and simply doesn't work for others. So for these "others", there's another protocol, which actually does make sure: it's called the transmission control protocol, or TCP. It's the one that makes sure your packets get delivered; and not only that, but in the right order, with no duplicates and no gaps. And it does so despite the fact that the packets themselves are still subject to the same unreliability as before: they can be dropped, duplicated or reordered along the way. What TCP does is turn this messy reality into a reliable, ordered stream of bytes between two processes — a connection.

And of course, it comes at a price: it requires a lot of back and forth. It's almost as if it has a dialogue with the other side, like "are you ready?", "yes", "OK, here's the first part. Did you get it?", "got it", "OK, now the second part". And because it's a dialogue, there are jokes about TCP jokes, because they keep repeating until you acknowledge them.

In fact, that's basically what happens: each side has a number, called a sequence number, which it uses to number the bytes it sends. The receiving side then acknowledges what it got, by sending back the number of the next byte it expects; and if the sender doesn't receive an acknowledgment within a reasonable amount of time, it retransmits. The receiving side also uses the sequence numbers to put everything in the right order, and discard any duplicates.

Before any of this can happen, the two sides need to synchronize their sequence numbers: this is called the three-way handshake. First, the client sends a SYN: "hello, I'd like to initiate a connection; my initial sequence number is going to be x"; then the server replies with a SYN-ACK: "hello! My initial sequence number is going to be y, and by the way, I got your x" — and the client sends an ACK: "got your y". Only then can data be transmitted — and when the connection is done, it's closed with a similar exchange, using FIN instead of SYN, or an abrupt RST instead of anything at all.

To see it for ourselves, let's look at another recording: this time, a browser logging in to a small web server running on port 8000 of the same machine. To record that, we need to use the loopback interface, and there's something unusual about its frames: they don't have an Ethernet header. Instead, they start with 4 bytes denoting the family of the address — that's what link type 108 means. Now, let's see if we can decode the TCP headers of the server's response:

>>> data = open('login.pcap', 'rb').read()
>>> struct.unpack('<I', data[20:24])
(108,)
>>> packets = [frame[4:] for timestamp, frame in read_pcap(data)]
>>> def tcp(packet):
...     segment = packet[(packet[0] & 0xf) * 4:]
...     source, destination, seq, ack, offset_flags = struct.unpack('!HHIIH', segment[:14])
...     payload = segment[(offset_flags >> 12) * 4:]
...     return source, destination, seq, ack, offset_flags & 0x3f, payload
>>> for packet in packets[1:8]:
...     source, destination, seq, ack, flags, payload = tcp(packet)
...     print(source, '->', destination, seq, len(payload), payload[:24])
8000 -> 53716 2140528801 17 b'HTTP/1.0 200 OK\r\n'
8000 -> 53716 2140528818 40 b'Content-Type: text/html;'
8000 -> 53716 2140528858 21 b'Content-Length: 282\r\n'
8000 -> 53716 2140528879 39 b'Server: Werkzeug/0.16.0 '
8000 -> 53716 2140528918 37 b'Date: Sun, 10 Nov 2019 1'
8000 -> 53716 2140528955 2 b'\r\n'
8000 -> 53716 2140528957 282 b'<!DOCTYPE html>\n<html>\n '

Walking through it, we notice that tcp skips the IP header, whose length is encoded in the low four bits of its first byte, then unpacks the ports and the sequence and acknowledgment numbers, and finally finds the payload after the TCP header, whose length is encoded in the top four bits of its offset field. If we look at the sequence numbers, we'll notice something interesting: the first one is 2140528801, and the next one is exactly 17 more, and the next one is 40 more: they literally count the bytes transmitted, and each segment starts exactly where the previous one ended.

But there's something else to notice: this server actually sent its response in 7 different pieces: the status line, each header, a blank line, and the body — each in a different segment. And this is important, because TCP doesn't have anything like a notion of "message boundaries": it's a stream, so the receiving party might read these bytes in 7 pieces, or in one piece, or any other way it likes to split them. Remember this, because it'll be the main problem of our next chapter.

Now, let's look at the flag bits in the client's request, and check the server's acknowledgment number:

>>> FLAGS = ['FIN', 'SYN', 'RST', 'PSH', 'ACK', 'URG']
>>> source, destination, seq, ack, flags, payload = tcp(packets[0])
>>> source, destination, bin(flags)
(53716, 8000, '0b11000')
>>> [name for i, name in enumerate(FLAGS) if flags >> i & 1]
['PSH', 'ACK']
>>> tcp(packets[1])[3] == seq + len(payload)
True

These are the bits in the low byte of the header: FIN, SYN, RST, PSH, ACK and URG. Our request has both PSH and ACK: and indeed, the server's next segment acknowledges exactly the sequence number of our request, plus its length. However, our capture actually starts in the middle of an existing connection, so we don't see the actual handshake — which is a shame, because it's pretty interesting.

You see, those initial sequence numbers, which we just glossed over as x and y, are random, and that's actually pretty important. In the early days of the internet, they were rather predictable, and this turned out to be a security vulnerability: an attacker, who could guess what sequence number you'd expect, could literally inject packets into your connection without even seeing it. But if the attacker was on the same network as you, it could simply see your packets (and sequence numbers) as they flew by. In fact, it could even pretend to be the server, and respond to your packets before the real one has a chance to — a so called TCP injection, or a way to hijack a session; or, it could just keep sending you RST packets, trying to reset your connection until one of them lands, and your long download is aborted.

The handshake itself is not immune to attacks, either: it can be abused in what's called a SYN flood, when someone sends the server thousands of SYN packets with spoofed addresses. The server, unaware that these are fakes, keeps each of these half-open handshakes in memory, and it can only handle so many at the same time — so it ends up ignoring the real clients, who can't get in. This is something to keep in mind when we call listen with some number in the next chapter: this number represents the size of this waiting room, if you will.

Names: DNS

There is, however, no protocol for talking to google.com. The internet is a network of machines, not websites; and machines have IP addresses, not names. Of course, nobody wants to type an IP address to search the web; so there's a side protocol that translates names to addresses, which happens before the actual conversation begins. It's called the domain name system, or DNS, and in Python it's as simple as this — for localhost, or for the website of python.org, on port 443:

>>> socket.gethostbyname('localhost')
'127.0.0.1'
>>> for family, kind, protocol, _, address in socket.getaddrinfo('python.org', 443, proto=socket.IPPROTO_TCP):
...     print(family.name, address)
AF_INET ('151.101.128.223', 443)
AF_INET ('151.101.64.223', 443)
AF_INET ('151.101.0.223', 443)
AF_INET ('151.101.192.223', 443)
AF_INET6 ('2a04:4e42:400::223', 443, 0, 0)
AF_INET6 ('2a04:4e42::223', 443, 0, 0)
AF_INET6 ('2a04:4e42:600::223', 443, 0, 0)
AF_INET6 ('2a04:4e42:200::223', 443, 0, 0)

The second call actually returns quite a lot of information about all the possible ways in which we can reach this service — it tells us its address family (AF_INET or AF_INET6), its socket type (SOCK_STREAM or SOCK_DGRAM), its protocol (IPPROTO_TCP or IPPROTO_UDP), and an address tuple. As we can see, python.org has four IPv4 addresses and four IPv6 addresses (and note that the IPv6 tuples have two extra fields: the flow info and the scope ID). A well-behaved client will attempt to connect to each of these addresses in order, until it succeeds — which is what socket.create_connection does for us. For localhost, we don't actually have to ask anyone; it resolves to 127.0.0.1, usually from the /etc/hosts file of our machine.

DNS is actually a big, complicated architecture: when our machine needs to resolve a name, it doesn't necessarily know the answer, so it asks some other machine — called a resolver — that it's configured to rely on. This resolver is usually provided by our internet service provider, which is what our router tells us to use for accessing the internet; or it can be a public resolver, like Google's 8.8.8.8 or Cloudflare's 1.1.1.1. If this resolver doesn't know the answer either, it will have to ask someone else, so it will climb up a hierarchy of servers: the root servers will point it to the servers responsible for .com domains, which will point it to the servers responsible for google.com, which should know the answer.

To give you another analogy, it's like a bureaucracy: you come to your boss to approve something — and if she has the authority, she approves it; but if she doesn't, she goes and asks her boss, and so on up the chain. To speed things up, every boss along the way might remember the answer for a while, so that the next time someone asks, they can reply immediately: a technique known as caching.

Now that we have some context, let's see what actually happens when we resolve a name. We already know it's a request over UDP; and if we look at its bytes, we'll see that the DNS query starts with a 12-byte header: an ID, some flags, and four 2-byte counters for the number of questions, answers, authority records and additional records, all of which is followed by the question:

>>> dns = frames[5][42:]
>>> ident, flags, questions, answers, authorities, additionals = struct.unpack('!6H', dns[:12])
>>> hex(flags), questions, answers
('0x100', 1, 0)
>>> def read_name(data, offset):
...     labels = []
...     while (length := data[offset]) != 0:
...         if length >= 0xc0:
...             pointer, = struct.unpack('!H', data[offset:offset + 2])
...             name, _ = read_name(data, pointer & 0x3fff)
...             return '.'.join(labels + [name]), offset + 2
...         labels.append(data[offset + 1:offset + 1 + length].decode())
...         offset += 1 + length
...     return '.'.join(labels), offset + 1
>>> read_name(dns, 12)
('www.facebook.com', 30)
>>> struct.unpack('!HH', dns[30:34])
(1, 1)

The flags, 0x100, mean "recursion desired": that is, we'd like this resolver to do the climbing for us. The question itself is encoded as a sequence of labels, each preceded by its length: 3 for www, 8 for facebook, 3 for com, and then a 0; and then the type (1, so A) and class (also 1, so internet) of record we want to retrieve — so, an A record, mapping a name to an IPv4 address, on the internet. Note that the length of a label can be 0xc0 or more, in which case it's not a length, but a pointer: the two bytes' low 14 bits are interpreted as an offset into the message, indicating a name that already appeared in it — a compression technique, and our read_name function accounts for that, following it recursively.

Now that we understand the question, let's see how the answer is encoded:

>>> dns = frames[6][42:]
>>> struct.unpack('!6H', dns[:12])
(12292, 33152, 1, 2, 0, 0)
>>> offset = 34
>>> for _ in range(2):
...     name, offset = read_name(dns, offset)
...     kind, cls, ttl, size = struct.unpack('!HHIH', dns[offset:offset + 10])
...     offset += 10
...     if kind == 5:
...         value, _ = read_name(dns, offset)
...     else:
...         value = ipaddress.ip_address(dns[offset:offset + size])
...     offset += size
...     print(name, kind, ttl, value)
www.facebook.com 5 3588 star-mini.c10r.facebook.com
star-mini.c10r.facebook.com 1 59 157.240.1.35

The reply has the same ID, and indicates that there is one question and two answers. The first one is called a CNAME, or "canonical name": an alias that indicates that www.facebook.com is really star-mini.c10r.facebook.com, and is valid for 3588 seconds. The second answer is what we want: an A record, mapping this name to the IP address 157.240.1.35, and valid for 59 seconds. These numbers, known as the TTL, or time to live, indicate how long answers can be cached for — and it's not the same for everything.

And that's how this whole process can be hijacked, known as DNS poisoning: DNS has no cryptography, so if we manage to get our answer in before the actual resolver does, and our answer is forged — for example, we say that google.com is 1.2.3.4 — then that's what the client will use to connect, and we didn't even have to touch any low-level protocols like TCP or IP. In other words, we didn't have to break into the post office — we could just change the address on the recipient's website, and any letters sent there will go elsewhere. Even worse, because of caching, such a poisoned resolver can serve this lie to anyone who asks — and in 2008, the Kaminsky attack made this kind of poisoning practical on resolvers that are out there on the open internet. DNSSEC is an attempt to fix this, but its adoption is still rather slow.

A more benign version of this you might have seen in action: when you connect to a hotel or airport WiFi, you might be redirected to a login page no matter which website you try to open — and that's usually because the WiFi's DNS server simply returns its own address for any name you want to resolve, until you actually provide this login.

HTTP: Text over TCP

So that's how the internet works. But what about the language of the web, that dialect in which clients and servers talk?

It's called HTTP, HyperText Transfer Protocol, and it's a very well-defined and very dichotomous client-server architecture. Clients ask servers for stuff, servers bring it to them, and that's it. And it's surprisingly simple: basically, it's a bunch of strings of text, being sent back and forth over TCP.

A client sends a request to a server, which has a method (like GET, meaning "bring me this", or POST, meaning "take this"), a path (that is, "what" — / is the index, /foo is the resource foo), and the protocol version (HTTP/1.0 or HTTP/1.1); then, a line break (CRLF, or \r\n); then, zero or more headers — that is, Name: value pairs of metadata, like which host you'd like (Host: www.python.org) or what language you prefer (Accept-Language: fr — that's how people from France get Google in French); then, an empty line, and an optional body.

A server returns a response, which has the protocol version, a status code and its reason (200 OK; 3xx for "it's over there", 404 for "not found", 500 for "I broke"), some headers, another empty line, and finally, the content — usually, a webpage in HTML for a browser to display, or some data in JSON for a script to consume.

If you want to see it for yourself, without any magic, we can start a tiny webserver from the standard library, in a background thread:

>>> import http.server, threading
>>> class Hello(http.server.BaseHTTPRequestHandler):
...     def do_GET(self):
...         body = b'Hello, world!\n'
...         self.send_response(200)
...         self.send_header('Content-Type', 'text/plain')
...         self.send_header('Content-Length', str(len(body)))
...         self.end_headers()
...         self.wfile.write(body)
...     def log_message(self, format, *args):
...         print('server:', format % args)
>>> server = http.server.HTTPServer(('127.0.0.1', 8000), Hello)
>>> threading.Thread(target=server.serve_forever, daemon=True).start()

This will start answering every GET request to http://localhost:8000/ with "Hello, world!", so let's break it down: do_GET is called whenever a GET request is made, and it simply sends a status line, two headers and a body. We've also overridden log_message to print out the message, just to see the server's side of things.

Now we can make a raw HTTP request to our local server over a plain socket, just to make sure we understand the situation:

>>> client = socket.create_connection(('127.0.0.1', 8000))
>>> client.sendall(b'GET / HTTP/1.1\r\nHost: localhost\r\n\r\n')
>>> chunks = []
>>> while chunk := client.recv(4096):
...     chunks.append(chunk)
server: "GET / HTTP/1.1" 200 -
>>> print(b''.join(chunks).decode())
HTTP/1.0 200 OK
Server: BaseHTTP/0.6 Python/3.13.9
Date: Wed, 14 Oct 2026 17:51:02 GMT
Content-Type: text/plain
Content-Length: 14

Hello, world!

Here, we connect, and send the request's bytes manually; then we receive the response until recv returns an empty bytes object, which means the server closed the connection. You can see the request line and status code being logged by the server; and the response's anatomy is just like we discussed. Note that this server actually speaks HTTP/1.0, which closes the connection after each response, and that's how we know it's over — in HTTP/1.1, connections are kept open, and we have to rely on Content-Length instead. This is what a browser does behind the scenes — there's no need for Chrome when all you need is a socket. And if we do the same to python.org on port 80:

>>> client = socket.create_connection(('python.org', 80))
>>> client.sendall(b'GET / HTTP/1.1\r\nHost: python.org\r\nConnection: close\r\n\r\n')
>>> response = client.recv(4096)
>>> print(response.decode())
HTTP/1.1 301 Moved Permanently
Connection: close
Content-Length: 0
Server: Varnish
Retry-After: 0
Accept-Ranges: bytes
Date: Wed, 14 Oct 2026 17:51:02 GMT
Via: 1.1 varnish
X-Served-By: cache-toj-leto2350045-TOJ
X-Cache: HIT
X-Cache-Hits: 0
X-Timer: S1792000263.703380,VS0,VE0
Location: https://www.python.org/
Strict-Transport-Security: max-age=315360000; preload

This time, the real python.org answers with 301 Moved Permanently, which means we need to go where its Location header says, https://www.python.org/ — so, all that plain HTTP on port 80 is used for is to redirect you to HTTPS, on port 443. You can also see the Via: 1.1 varnish and X-Cache: HIT headers, which indicate the answer was served from a CDN cache, not from the servers of python.org itself.

To understand why the whole internet moved to HTTPS, go back and look at that login recording again:

>>> payload = tcp(packets[16])[-1]
>>> payload.split(b'\r\n')[0]
b'POST /login HTTP/1.1'
>>> payload.split(b'\r\n\r\n')[1]
b'username=alice&password=123456'

The third connection's request is a POST to /login, with the body username=alice&password=123456. This is what you send, in plaintext, when you try to login to a website: anyone who can capture this traffic, like someone on the same WiFi with a NIC in promiscuous mode, or your ISP, or any router on the way, can see this text, and get your username and password. That's why HTTPS, which is exactly like HTTP but wrapped in TLS, an encryption layer that runs between TCP and HTTP, was developed: so that all they'd see would be noise.

Libraries like requests, or even the standard library's urllib, are just an abstraction layer over that, where they construct the request text for you, and parse the response into a status, a dictionary of headers, and the content. But we'll take the HTTP protocol apart properly when we talk about web development.

Dumb Networks, Smart Edges

Let's take a step back, and admire what we just built. At each layer, we solved a small problem and let the next one forget about it: the physical layer knows how to signal bits on a wire, the link layer knows how to deliver them within a local network, IP knows how to deliver them across networks, and TCP and UDP know how to deliver them to specific processes. On top of that, application protocols do whatever they want; and each layer can be swapped out independently: going from Ethernet to WiFi didn't mean we had to change anything for TCP.

The shape of this architecture is often compared to an hourglass: a lot of different link technologies at the bottom, a lot of different applications at the top, and a single, narrow passage in the middle — that's IP, the Internet Protocol. Everything is built on top of IP, and IP is built on top of everything — which is why we could connect networks that weren't even coordinated in advance.

Another interesting point about this design is known as the end-to-end principle — it was articulated by Saltzer, Reed and Clark in 1984, and states that functions like reliability or encryption should be implemented at the endpoints — in other words, the applications that actually care about these properties. The reason, they argued, is that these functions can only be implemented completely and correctly at these endpoints, and the network in the middle should be kept simple. So we have a dumb network that does its best, and smart edges: the reliability of TCP, for example, is actually a feature of the operating systems at the two ends, not of the routers. It's not that the post office cares whether you got your letter, or whether you were happy with it; but if you do care, you can always send it registered — and this is an arrangement between the sender and the receiver, which they can establish on top of an otherwise mundane delivery service.

This also explains why the internet grew so easily: anyone could come up with a new application, and simply do it — the network didn't have to give its permission. In fact, this is how the World Wide Web was invented by Tim Berners-Lee at CERN in 1989, long after the internet was already there; and you know what, it didn't require changing a single router to support this new fad.

There's an interesting, related principle called the robustness principle, attributed to Jon Postel: be conservative in what you send, and liberal in what you accept. It meant that the early, incompatible implementations could work together; but it's also how bugs and quirks become permanent, how parsers become an attack surface. Which brings us to the bill: the internet wasn't developed with security in mind. It started as a network of research institutions that trusted each other, and we saw how at every layer, it breaks: people can snoop on the local network and poison its ARP cache, spoof source addresses at the IP layer, inject packets and SYN flood at the TCP layer, poison DNS and send plaintext passwords over HTTP. The fixes, mostly by using cryptography, were added on later, and mostly at the edges, just like the end-to-end principle would suggest.

Going back to our main point — that the internet is not magic — you have to admit that it's not: it's just bytes, peeled layer by layer by programs that agree on formats, which we could read, quite literally, with struct and a few slices. In practice, as programmers, we're not likely to touch any of this: the networking stack is implemented by the operating system, and exposed through a single interface known as the socket. You just import socket, connect, and send; the operating system allocates a port, chooses a route, finds the gateway's MAC address and puts the bits on the wire.

In the next chapter, we're going to talk about this interface. We'll see how to write clients and servers with sockets, what exactly does it mean that they're blocking, why recv(4) might return less than four bytes, and even build a little protocol of our own on top of TCP.