Writing/The Unbearable Lightness of Coding
Chapter 29 · Draft · 45 min read
Web Development: How the Web Works
Google Is Just a Process on Port 80
Looking at the big picture, we built a whole stack over the last few chapters, from the bottom up: we started with packets and TCP, then moved to sockets, and then to servers that could juggle multiple clients, either with threads or with async. And while it's all nice and dandy, it doesn't actually say what these bytes that we're moving around mean.
The World Wide Web is one answer to this question — and by far the most successful one, at that. A lot of people seem to think of it as something magical, that just works — but really, if you go to Google, it's a process running on one of Google's servers, which opened a socket on port 80 (or 443 these days); and when a connection arrives, it accepts it, receives a piece of text called a request, runs the Google code to figure out an answer, and writes a piece of text called a response back on this socket. That's it — that's the whole web, in a nutshell.
To be fair, the web is not the same thing as the internet; the internet is the mesh of streets connecting different buildings, so we can get from one place to another — but it's hard to navigate this city without street names, signage and a map. The World Wide Web is the project that painted the street names, put up the signage, and created a map over this internet. Tim Berners-Lee, who was working at CERN in 1989-1990, had an idea: all this information on the internet could be thought of as resources, and every resource could have a unique identifier, called a URI. He then proposed a request-response architecture, where clients would want resources, and servers would have them; and he invented hypertext — text annotated with a URI, so that when you clicked it, a request for this resource would be sent. He also went ahead and implemented the first web server and web browser, so he definitely earned the title of "inventor of the World Wide Web".
Nowadays, all of this is so trivial that we get annoyed when a word on the screen isn't clickable — but back then, it was completely revolutionary. So let's prove my previous point with nothing but the tools from the previous chapters; no browser, no library, just a raw socket to www.python.org on port 80, and a few lines of text:
>>> import socket
>>> connection = socket.create_connection(('www.python.org', 80))
>>> connection.sendall(b'GET / HTTP/1.1\r\nHost: www.python.org\r\nConnection: close\r\n\r\n')
>>> print(connection.recv(4096).decode())
HTTP/1.1 301 Moved Permanently
Connection: close
Content-Length: 0
Server: Varnish
Retry-After: 0
Location: https://www.python.org/
Accept-Ranges: bytes
Date: Wed, 14 Oct 2026 17:47:28 GMT
Via: 1.1 varnish
X-Served-By: cache-toj-leto2350047-TOJ
X-Cache: HIT
X-Cache-Hits: 0
X-Timer: S1792000048.312450,VS0,VE0
Strict-Transport-Security: max-age=63072000; includeSubDomains; preload
What we get is also text — the first line is called a status line (HTTP/1.1 301 Moved Permanently), then there's a series of Key: Value headers, and finally an empty body. The Location header tells us where to go instead: the same page, but over https. This is called a redirect, and it's something your browser does silently, every time you type a bare domain name. You can also see Server: Varnish and X-Cache: HIT, which means we didn't even get the response from python.org's own webserver, but from a cache in front of it — we'll see later in the chapter why this is a thing in HTTP.
As for https, it's not a different language, just the same text over an encrypted TLS tunnel. Luckily, Python's ssl module can wrap an ordinary socket in such a tunnel, and then we speak exactly as before:
>>> import ssl
>>> context = ssl.create_default_context()
>>> connection = context.wrap_socket(socket.create_connection(('www.python.org', 443)), server_hostname='www.python.org')
>>> connection.sendall(b'GET / HTTP/1.1\r\nHost: www.python.org\r\nConnection: close\r\n\r\n')
>>> response = b''
>>> while chunk := connection.recv(4096):
... response += chunk
>>> head, _, body = response.partition(b'\r\n\r\n')
>>> print(head.decode().splitlines()[0])
HTTP/1.1 200 OK
>>> b'content-encoding: gzip' in head
True
>>> import gzip
>>> print(gzip.decompress(body)[:70].decode())
<!doctype html>
<html class="no-js" lang="en" dir="ltr">
<head>
So we've established a secure TLS connection, sent the same thing, and received something that looks similar — but there's a bit more work this time. First, we had to read until the connection was closed, which we told the server to do when it's done (in Connection: close). Then, we had to separate the head from the body, which is indicated by the first empty line (\r\n\r\n); and even then, the body itself turned out to be compressed with gzip, even though we didn't ask for it — but that's something the standard allows, where not specifying an Accept-Encoding header means any encoding is acceptable. All that decompressed, it's just HTML: the source of the page that your browser ends up drawing.
That's all there is to a browser — and for the rest of this chapter, we'll elaborate on this idea: how resources are named (URLs), the grammar of requests and responses (HTTP), what a page is made of (HTML, CSS and Javascript), how a page load unfolds, how a protocol with no memory can still remember who you are (cookies), and how machines can talk to machines (REST APIs). In the next chapter, we'll implement the server side ourselves.
As an aside, whenever people start working on the web, they're often overwhelmed with the amount of different technologies involved. It's a bit like going to study science, and realizing there's biology and physics and chemistry — whole other disciplines that are pretty unrelated, done by pretty different people. I like to think of this chapter as a tasting course, just enough to see how it all fits together.
URLs: Finding Resources
And so, we arrive at resources. Every resource has a unique identifier, known as a Uniform Resource Identifier, or URI. But being unique is not enough: the number on my ID card is unique, at least in Israel, but it's not very helpful in finding me. What we need is a Uniform Resource Locator, or URL: a set of instructions on how to get to a particular resource. In practice, the two concepts overlap a lot (if we know how to describe how to get something, why not use it as its name?), so I'll use them interchangeably.
The URL's anatomy, in its most general form, looks like this: scheme://username:password@host:port/path?query#fragment. The scheme indicates which language we're speaking: http, https, ftp, ssh and so on. The host indicates who we're talking to, which is some DNS name we discussed in the networking chapter; the port is the virtual door on that host that we want to knock on, and defaults to 80 for http and 443 for https. Then, the path indicates which resource we want, and is organized like a filesystem; the query lets us pass some parameters; and the fragment lets us point to a particular place within that resource.
We can parse a URL to break it down into its constituent parts:
>>> from urllib.parse import urlparse
>>> url = urlparse('https://dan:1234@books.example.com:8443/python/ch28.html?lang=en&page=2#cookies')
>>> url
ParseResult(scheme='https', netloc='dan:1234@books.example.com:8443', path='/python/ch28.html', params='', query='lang=en&page=2', fragment='cookies')
>>> url.hostname, url.port, url.username, url.password
('books.example.com', 8443, 'dan', '1234')
Walking through this, the first thing you'll notice is that it returns a named tuple with six fields. The netloc field is simply the "who" part; and then the hostname, port, username and password attributes further parse it (with the port being an integer). The params field is a weird holdover from an old way of writing paths, and is almost always empty. As for credentials in a URL, it's actually legal (like ssh://dan:1234@... in the lectures), but not really recommended (unless you're sure nothing is being logged, shared or displayed in an address bar).
Here's a tiny exercise from one of my courses: a function that receives a URL, and returns the protocol, the domain and the path:
>>> def parse_url(url):
... result = urlparse(url)
... return result.scheme, result.netloc, result.path
>>> parse_url('http://www.google.com/search?q=infosec')
('http', 'www.google.com', '/search')
The query string is used to parametrize a resource: /search?q=infosec means, "I'd like Google's search resource, and I'd like to parametrize it with q=infosec". Of course, everyone gets the same resource — but it can be parametrized to display different results for different queries.
The query string itself can be parsed further:
>>> from urllib.parse import parse_qs, urlencode
>>> parse_qs('q=python+web&lang=en&lang=he')
{'q': ['python web'], 'lang': ['en', 'he']}
>>> urlencode({'q': 'python web', 'page': 2})
'q=python+web&page=2'
Notice how values are returned as lists, since a parameter might repeat; and how a + in a query string is translated into a space, as well as how to do the opposite. We'll see this again when we talk about requests, which accepts a params=, and does this for us.
Another note about URLs: since they're limited to a small subset of ASCII characters, anything that doesn't fit this mold needs to be encoded in some way. The standard way to do that is by percent-encoding: replacing each byte of its UTF-8 representation with % and its two hexadecimal digits. This is what's happening if you ever copied a URL from the Hebrew Wikipedia, and got a monstrous string full of percent signs:
>>> from urllib.parse import quote, unquote
>>> quote('/wiki/פייתון')
'/wiki/%D7%A4%D7%99%D7%99%D7%AA%D7%95%D7%9F'
>>> unquote('%D7%A4%D7%99%D7%99%D7%AA%D7%95%D7%9F')
'פייתון'
Since each Hebrew letter takes two bytes in UTF-8 (see our discussion of strings and bytes in the Native Types chapter), the six letters of the word become a total of twelve %XX triplets. unquote reverses the process; and the browser actually displays the decoded version, while sending the encoded one.
There are some more interesting nuances to URLs, like the fact that links are often relative — in which case, the browser resolves them relative to the URL of the page it's on, much like we resolve a relative path against the current directory:
>>> from urllib.parse import urljoin
>>> urljoin('http://example.com/books/python/ch28.html', 'ch29.html')
'http://example.com/books/python/ch29.html'
>>> urljoin('http://example.com/books/python/ch28.html', '../javascript/index.html')
'http://example.com/books/javascript/index.html'
>>> urljoin('http://example.com/books/python/ch28.html', '/about.html')
'http://example.com/about.html'
>>> urljoin('http://example.com/books/python/ch28.html', '#cookies')
'http://example.com/books/python/ch28.html#cookies'
That is, a bare name replaces the last path segment; ../ climbs up a level; a leading / stays on the same host, but starts from its root; and a fragment-only link stays on the same page entirely. And a nice detail to know, while we're on the subject, is that the fragment is never sent to the server — it's kept by the browser to scroll to the appropriate element (or to let Javascript read it, if it wants to).
HTTP by Hand
Now that we know what to ask for, let's talk about how to ask for it. The HyperText Transfer Protocol, or HTTP, is surprisingly simple, and very much a client-server language: clients ask, and servers answer. That's really it; that's the protocol, and my whole point is that it's not a big deal: just strings being sent back and forth.
A request starts with a request line, followed by zero or more header lines of the form Key: Value, followed by an empty line, followed by an optional body. The request line has three parts: a method, which is the action we'd like to do, like GET or POST, a target, which is the path of the resource we'd like to work on, and a protocol version. All lines are terminated with \r\n.
A response is the same, except that its first line is a status line: a protocol version, a numeric status code, and a reason phrase, all separated by spaces. Again, all lines are terminated with \r\n.
The first websites were completely static: a website was, in essence, a remote file system. If you wanted /index.html, the server would read that file from its disk, and send you the result; your browser was, essentially, a distributed, visual file system. We can even start one such server with Python's standard library: just run python -m http.server on our site directory, and it will start serving files from there. This is what its terminal will look like:
$ python -m http.server 8000 --bind 127.0.0.1 --directory site
Serving HTTP on 127.0.0.1 port 8000 (http://127.0.0.1:8000/) ...
127.0.0.1 - - [14/Oct/2026 19:47:35] "GET /hello.txt HTTP/1.0" 200 -
127.0.0.1 - - [14/Oct/2026 19:47:35] code 404, message File not found
127.0.0.1 - - [14/Oct/2026 19:47:35] "GET /nope.txt HTTP/1.0" 404 -
127.0.0.1 - - [14/Oct/2026 19:47:36] code 501, message Unsupported method ('POST')
127.0.0.1 - - [14/Oct/2026 19:47:36] "POST /login HTTP/1.0" 501 -
That is, it's the server's terminal, and those lines are the requests we're about to make from a different terminal — one per request, with the client's address, time, request line, and status code.
Let's start with a raw request: I'm using HTTP/1.0, which is a simpler version, where the server answers one request per connection and closes it, so we don't have to worry about our reads getting stuck:
>>> import socket
>>> connection = socket.create_connection(('127.0.0.1', 8000))
>>> connection.sendall(b'GET /hello.txt HTTP/1.0\r\nHost: localhost:8000\r\n\r\n')
>>> print(connection.recv(4096).decode())
HTTP/1.0 200 OK
Server: SimpleHTTP/0.6 Python/3.13.9
Date: Wed, 14 Oct 2026 17:47:35 GMT
Content-type: text/plain
Content-Length: 14
Last-Modified: Wed, 14 Oct 2026 17:25:31 GMT
Hello, world!
OK, let's go over it. We got 200 OK, and a Server header saying who answered us; then a Content-type, which is a guess based on the file's extension; then a Content-Length, which tells us the exact size of the body (which is how a client knows when it's over, if the connection stays open); a Last-Modified header, which caches can use to ask "has this changed since this time?"; and finally, the empty line and the actual file.
Now, let's see what happens if the file doesn't exist:
>>> def request(raw):
... connection = socket.create_connection(('127.0.0.1', 8000))
... connection.sendall(raw)
... response = b''
... while chunk := connection.recv(4096):
... response += chunk
... connection.close()
... return response.decode()
>>> print(request(b'GET /nope.txt HTTP/1.0\r\n\r\n'))
HTTP/1.0 404 File not found
Server: SimpleHTTP/0.6 Python/3.13.9
Date: Wed, 14 Oct 2026 17:47:35 GMT
Connection: close
Content-Type: text/html;charset=utf-8
Content-Length: 335
<!DOCTYPE HTML>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Error response</title>
</head>
<body>
<h1>Error response</h1>
<p>Error code: 404</p>
<p>Message: File not found.</p>
<p>Error code explanation: 404 - Nothing matches the given URI.</p>
</body>
</html>
I wrote a little request helper here, which just reads until the server closes the connection — but note that we get 404 File not found, and there's a body. The status code is for machines; the body is for humans, and you can see it's an HTML error page. Also, note that our request didn't have a Host header at all: this was allowed in HTTP/1.0, but not in HTTP/1.1, because a single IP address can host multiple websites (that is, virtual hosting), and the only way the server can tell which of them you want is through the Host header.
Anyway, GET is for reading; POST is for writing, so it usually includes a body (just like when we write to a file, we, well, write to it). We "write to" the /login resource, and a Content-Length header tells how long what we're writing is. I once had the example on my slide be a login with username=dan&password=1234 in the body, and a student immediately asked me: "in plaintext?" — and I said, yeah, in plaintext, that's why we have HTTPS.
>>> form = b'username=dan&password=1234'
>>> print(request(b'POST /login HTTP/1.0\r\nContent-Type: application/x-www-form-urlencoded\r\nContent-Length: %d\r\n\r\n%s' % (len(form), form)).splitlines()[0])
HTTP/1.0 501 Unsupported method ('POST')
So we get 501 Not Implemented, "Unsupported method ('POST')": a static file server has no idea what to do with a request to "write to" /login. It's not a file, it's code; and that's what the dynamic web is all about, which we'll get to (and write ourselves!) in the next chapter. For now, note that from the shell, we can do the same with curl -i, or just ask for the headers with curl -I; and that's really all that browsers and other HTTP libraries are: very elaborate versions of our little request helper, which open a connection, send text, receive text, and parse it.
Requests, Statuses and Headers
Of course, it's not very fun to parse responses by hand, so libraries are here to help. HTTP is a layer of abstraction over TCP, and these libraries add another layer of abstraction over it: in Python, you can use the standard urllib.request module, or the much more popular 3rd party requests library, whose tagline is "HTTP for Humans".
For the rest of this chapter, I'll use a small local server that I wrote for this purpose (it's localweb.py in this chapter's code). It serves the static files from before, mimics httpbin.org by echoing back whatever it receives, and has a toy login and a toy JSON API. How to write such a server, by the way, is exactly what we'll cover in the next chapter:
>>> import requests
>>> response = requests.get('http://localhost:8000/hello.txt')
>>> response
<Response [200]>
>>> response.status_code, response.reason
(200, 'OK')
>>> response.headers['content-type']
'text/plain'
>>> response.text
'Hello, world!\n'
The Response object has its status code as part of its representation; and indeed, the status line is made available in status_code and reason, while the headers are parsed into a case-insensitive dictionary (since header names are case-insensitive in HTTP, so 'content-type' would work as well). The body is in text, decoded to a string (or in content, if you want the raw bytes). Let's do a GET with some query parameters, and add a custom header:
>>> response = requests.get('http://localhost:8000/anything', params={'q': 'python web'}, headers={'Accept-Language': 'fr'})
>>> response.url
'http://localhost:8000/anything?q=python+web'
>>> data = response.json()
>>> data['args']
{'q': ['python web']}
>>> data['headers']
{'Host': 'localhost:8000', 'User-Agent': 'python-requests/2.34.2', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Accept-Language': 'fr'}
And note that this also sends the parameters as a query string, which we built by hand before (requests uses urlencode, which we've seen in the previous section). The echo also reveals some other headers requests sends by default: the Host, User-Agent (that's how the server knows what kind of client you're using; browsers send long strings that start with Mozilla for historical reasons); Accept-Encoding: gzip, which is why compressed bodies are the norm; and a few other things, including Connection: keep-alive, since HTTP/1.1 is all about reusing the same connection for multiple requests.
These headers are all metadata that the server can act on: if you browse to Google from France, you get a result in French, because your browser (configured by your operating system) automatically adds Accept-Language: fr to every request, and Google takes this into account.
>>> response = requests.post('http://localhost:8000/anything', data={'x': 1, 'y': 2})
>>> response.json()['form']
{'x': ['1'], 'y': ['2']}
>>> response.request.headers['Content-Type']
'application/x-www-form-urlencoded'
>>> response.request.body
'x=1&y=2'
And notice how requests does all the heavy lifting for us: data= is urlencoded and sent as a body with Content-Type: application/x-www-form-urlencoded, just like a browser would when you submit an HTML form; if it was json=, it would be serialized to a JSON body and set Content-Type to application/json instead. And if you ever need to do something more sophisticated, like a timeout, it's all there, in one place.
Status codes are another part of the standard that are worth knowing. They come in families: 2xx means everything is OK, 3xx means "go look elsewhere", 4xx means something is wrong with the client request, and 5xx means something is wrong with the server. Some of the ones worth knowing are 200 OK, 201 Created (when a resource was created successfully), 204 No Content (since a 200 is supposed to have a body, and there's nothing to return); 301 and 302 for redirects, 403 Forbidden, 404 Not Found, 405 Method Not Allowed (the resource exists, but not for this particular verb), 409 Conflict, 500 Internal Server Error (the server crashed), 502 Bad Gateway (a proxy in front of the server got nonsense from the server behind it); and 418 I'm a Teapot, from an April Fools' RFC (2324, the Hyper Text Coffee Pot Control Protocol, 1998) — so even the internet has a sense of humor.
>>> requests.get('http://localhost:8000/status/204')
<Response [204]>
>>> requests.get('http://localhost:8000/status/418')
<Response [418]>
>>> from http import HTTPStatus
>>> HTTPStatus(418), HTTPStatus(418).phrase
(<HTTPStatus.IM_A_TEAPOT: 418>, "I'm a Teapot")
>>> [f'{status.value} {status.phrase}' for status in HTTPStatus if status.value in (200, 201, 204, 301, 302, 403, 404, 405, 409, 500, 502)]
['200 OK', '201 Created', '204 No Content', '301 Moved Permanently', '302 Found', '403 Forbidden', '404 Not Found', '405 Method Not Allowed', '409 Conflict', '500 Internal Server Error', '502 Bad Gateway']
>>> response = requests.get('http://localhost:8000/status/404')
>>> response.ok
False
>>> response.raise_for_status()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
...
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http://localhost:8000/status/404
Notice, by the way, that requests doesn't raise an exception on a 404 — it's a perfectly valid response to a perfectly valid request. ok is False for 4xx and 5xx, so if you want to raise an error on anything other than 2xx, raise_for_status() is there for you; and the http.HTTPStatus enum in Python knows all of them, with their standard phrases.
As a sidenote, it's interesting to compare requests to the standard urllib.request module:
>>> import json, urllib.parse, urllib.request
>>> data = urllib.parse.urlencode({'x': 1, 'y': 2}).encode()
>>> with urllib.request.urlopen('http://localhost:8000/anything', data) as response:
... json.loads(response.read().decode())['form']
{'x': ['1'], 'y': ['2']}
>>> urllib.request.urlopen('http://localhost:8000/status/404')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
...
urllib.error.HTTPError: HTTP Error 404: Not Found
For one, we have to do a POST with urlopen (it's a hallucination of an interface, really: you either pass it one argument to make a GET request, or two to make a POST request). Then, we have to encode our form ourselves; and then read, decode and json.loads it ourselves; and if we get a 404, it raises an exception.
It gets even worse when we need authentication: the way to do it in urllib is with an HTTPPasswordMgrWithDefaultRealm, an HTTPBasicAuthHandler and build_opener, and I've worked with HTTP a lot and never once needed a realm — in requests, it's simply auth=(username, password). Remember the lesson from the Workflow chapter, about easy things being easy? This is the same lesson, applied to an HTTP client.
HTML, CSS and Javascript
Let's talk about what makes a web page.
The first thing you need to understand is that the body of a response can be anything, really: text, or an image, or a video; and the Content-Type header tells the browser what to expect (these are called MIME types: text/plain, image/svg+xml or text/html). So when you request a web page, you expect the body to be HTML, which is a set of instructions for the browser as to how it should be rendered.
In other words, we're talking about three different technologies, and three different jobs: HTML, or HyperText Markup Language, is in charge of the content and its structure: this is the title, this text goes here, and that image there. CSS, or Cascading Style Sheets, is in charge of the presentation: how fonts and sizes and colors look, and how the overall layout is organized. And then there's Javascript, which is in charge of the behavior — what happens when the user does something.
Historically, HTML was used to do all of the above, with different tags that specified fonts and colors and the like. But then it was decided that it's better to decouple the style from the structure, so that the two could be developed and changed independently — so here we are.
Let's take a look at the HTML of our sample site:
$ cat site/index.html
<!DOCTYPE html>
<html>
<head>
<title>The Unbearable Lightness of Coding</title>
<link rel="stylesheet" href="style.css">
</head>
<body>
<h1>Hello, world!</h1>
<p>A book about the <a href="/about.html">unbearable lightness</a> of coding.</p>
<img src="logo.svg" alt="logo">
<button id="greet">Click me</button>
<script src="script.js"></script>
</body>
</html>
HTML is made up of tags, which are enclosed in angle brackets, and nest within each other. The root tag is always html, and it contains two parts: head, which contains metadata, like the title that's shown on the browser tab, and a link to a stylesheet; and body, which contains the actual content — in this case, a heading, a paragraph with an a (anchor) tag, which specifies a hyperlink via its href attribute, an img tag for an image, a button with an id, and a script tag. Note that some tags, like img and link, are void: they have no closing tag.
Also note how much of this page is actually about other resources: it's not one file, but a small graph of them, which the browser needs to fetch and understand how to display together. HTML5 also added semantic tags, like header, nav, article and footer, to specify what each part is about; it doesn't matter for how it looks, but it matters for screen readers, search-engine crawlers that determine how your page ranks, or link previews on social media. In fact, accessibility is arguably the most important reason to care about semantic tags at all.
For CSS:
$ cat site/style.css
body {
font-family: sans-serif;
background-color: #f0e2c8;
}
a {
color: #629ebb;
}
#greet {
background-color: red;
color: white;
}
A CSS file is made up of rules, which have a selector and some declarations. In this case, we select every body element, every a element, and the element whose id is greet, and specify that their font should be so and so, their color should be so and so, and so on. The cascading part refers to the fact that several rules can apply to a single element, in which case the more specific selector wins (an id is more specific than a tag name); and in case of a draw, the later rule wins. Some properties are also inherited from the parent element, so the exact calculation is somewhat complicated, but that's the general idea.
There are three ways to specify CSS: a style attribute on the element, a <style> tag in the page, or an external file via a <link> tag, which is the best way to reuse and cache it. (The colors are those of my course's website, whose slides were the website.)
CSS is also non-Pareto: usually, 80% of the effects can be achieved with 20% of the cause, but here, getting the last 20% right — that is, making everything pixel perfect, accounting for all the browser quirks, and having a responsive layout that adapts to every screen size — actually takes 80% of the effort. For programmers, it's OK to use frameworks like Bootstrap: it doesn't even pretend to be beautiful, it only pretends to be convenient. And the CSS Zen Garden shows that this separation is indeed possible, displaying the same HTML file styled in hundreds of wildly different ways.
For Javascript:
$ cat site/script.js
const button = document.getElementById('greet');
button.onclick = function () {
alert('Hey there!');
const p = document.createElement('p');
p.textContent = 'You clicked at ' + new Date().toLocaleTimeString();
document.body.appendChild(p);
};
This code selects the button with getElementById, and sets a function on its onclick property, which is a callback: the browser runs an event loop, like we saw in the async chapter, and whenever the user clicks the button, this function is invoked. It pops out an alert dialog and then creates a new paragraph element, appending it to the page even as it's being displayed. It's pretty crazy, if you think about it: a remote server sends you some code, and your browser just runs it on your machine. Applications like Google Docs or Facebook wouldn't be possible without it; and if this sounds dangerous, that's because it is — which is why web security is an interesting field of its own, full of browser sandboxes, the same-origin policy, and cross-site scripting.
Historically, in 1995, the Netscape company, which made the first major commercial browser, wanted to put Scheme (a dialect of Lisp) in the browser, so they hired Brendan Eich to do so. However, the management decided that it would be better to do something that looks like Java, the new hotness, which immediately ruled out Scheme (and also, funnily enough, languages like Perl, Tcl and Python; imagine a world where the language of the web is Python). So Brendan wrote a new language in about ten days — first called Mocha, then LiveScript, and eventually Javascript (to ride the hype of Java), whose standard is called ECMAScript.
This language is dynamically and weakly typed (whereas Python is dynamically but strongly typed: '1' + 1 throws a TypeError in Python, and results in '11' in Javascript). It's also extremely functional in nature, with callbacks and anonymous functions all over the place — which just so happened to be perfect for the asynchronous nature of the web. In fact, it reminds me of a story about languages: Esperanto was specifically designed to be simple and logical — and almost nobody speaks it; J. R. R. Tolkien, a philologist, invented Elvish, and then felt that it didn't make sense to have a language that didn't have a history and a people that spoke it, so he invented Middle-earth to go along; languages, like any other medium, grow organically, and accumulate a lot of "baggage" as a result — Javascript is no different (in fact, it has so much of it that I'm not even going to get into it). What I like less is that it doesn't have a BDFL — like Guido or Linus — so different browsers implemented things differently, which is why programming the web is easy; programming the web well is hard.
The DOM, and How a Page Load Unfolds
Now, browsers don't actually store webpages as text. When a page arrives, they immediately parse it into a tree of nodes, known as the Document Object Model, or DOM. They then draw this tree on the screen; and if some Javascript wants to access this DOM and modify it, they certainly don't give it the original text — they let it interact with this tree instead.
So when we say Javascript can change HTML, what we actually mean is that it can change this tree; and whenever we call a function like getElementById, what we actually mean is that we query this tree. To give another example, you might have noticed that I never mentioned how CSS is applied to HTML. The truth is, it's all part of this tree: and when some CSS says p { color: red; }, it's the browser that goes ahead and matches all the p nodes to this rule, coloring their text red as it paints them on our screen.
We can parse HTML ourselves, too. In Python, it's as easy as html.parser: it's an event-based parser, which means as it goes over the file, it calls some function (which we need to define) for every event it encounters, be it a starttag, an endtag, or some data. It's similar to the callbacks we've seen in the async chapter, only this time it's a class; so all we need to do is subclass it, and we'll be able to print the tree like so:
>>> from html.parser import HTMLParser
>>> class TreePrinter(HTMLParser):
... VOID = {'link', 'img', 'meta', 'br', 'input'}
... def __init__(self):
... super().__init__()
... self.depth = 0
... def handle_starttag(self, tag, attrs):
... print(' ' * self.depth + tag + (f' {dict(attrs)}' if attrs else ''))
... if tag not in self.VOID:
... self.depth += 1
... def handle_endtag(self, tag):
... self.depth -= 1
... def handle_data(self, data):
... if data.strip():
... print(' ' * self.depth + repr(data.strip()))
>>> import requests
>>> html = requests.get('http://localhost:8000/').text
127.0.0.1 - - [14/Oct/2026 19:47:43] "GET / HTTP/1.1" 200 -
>>> TreePrinter().feed(html)
html
head
title
'The Unbearable Lightness of Coding'
link {'rel': 'stylesheet', 'href': 'style.css'}
body
h1
'Hello, world!'
p
'A book about the'
a {'href': '/about.html'}
'unbearable lightness'
'of coding.'
img {'src': 'logo.svg', 'alt': 'logo'}
button {'id': 'greet'}
'Click me'
script {'src': 'script.js'}
Notice how the indentation is a result of the nesting, and how void elements like link and img don't have an end tag, so they don't open a level of indentation; also, note that text is treated as a node of its own, and all the attributes are exactly what getElementById and CSS selectors look things up by. This is the tree that the previous section's code was searching through and appending to.
And just to underline how fundamental this structure is, let me tell you how you can hack a website's DOM yourself, live, as you browse it. Open your browser's developer tools (if you're using Chrome, it's as simple as F12, or control+shift+I), go to google.com and select the doodle in the middle. You can actually delete it, or create a list and append it to the page, or whatever else you want. It's a neat party trick — but more than that, it's your own copy of the tree, so whatever you do, nobody else will see, and it will disappear when you refresh the page.
Now that we know what happens after a webpage arrives, let's see what actually happens when you type in a URL and hit enter. First, the browser parses the URL. It uses DNS to resolve the host name to an IP address, then establishes a TCP connection with it — and if it's a secure connection, does a TLS handshake as well. Then, it sends a GET request for the particular resource; and as the HTML starts arriving, the browser goes ahead and parses it, which reveals references to other resources: stylesheets, scripts, images, and so on — and the browser goes ahead and requests those, too, sometimes several at a time, over several connections. At this point, some scripts might start running, and the page becomes interactive; but it might also continue making requests in the background, even as you interact with it.
Let's be a (very dumb) browser ourselves: we fetch a webpage, parse its HTML, and find out what else we need to fetch in order to load it completely. Here's a load function that does that:
>>> from urllib.parse import urljoin
>>> class ResourceFinder(HTMLParser):
... def __init__(self):
... super().__init__()
... self.resources = []
... def handle_starttag(self, tag, attrs):
... attrs = dict(attrs)
... if tag == 'link' and attrs.get('rel') == 'stylesheet':
... self.resources.append(attrs['href'])
... elif tag in ('script', 'img') and 'src' in attrs:
... self.resources.append(attrs['src'])
>>> def load(url):
... response = requests.get(url)
... finder = ResourceFinder()
... finder.feed(response.text)
... for resource in finder.resources:
... sub = requests.get(urljoin(url, resource))
... print(f'{resource}: {sub.status_code}, {len(sub.content)} bytes, {sub.headers["Content-Type"]}')
>>> load('http://localhost:8000/')
127.0.0.1 - - [14/Oct/2026 19:47:44] "GET / HTTP/1.1" 200 -
127.0.0.1 - - [14/Oct/2026 19:47:44] "GET /style.css HTTP/1.1" 200 -
style.css: 200, 151 bytes, text/css
127.0.0.1 - - [14/Oct/2026 19:47:44] "GET /logo.svg HTTP/1.1" 200 -
logo.svg: 200, 117 bytes, image/svg+xml
127.0.0.1 - - [14/Oct/2026 19:47:44] "GET /script.js HTTP/1.1" 200 -
script.js: 200, 258 bytes, text/javascript
If we look at the server's log, we'll see that it actually receives 4 requests for what we might consider a single page: and real websites load dozens, if not hundreds of additional resources for fonts, ads and analytics. Once again, we use urljoin to resolve the paths, like we did before; and you might have noticed that every response has its own Content-Type, which is how the browser knows whether to apply the CSS, run the Javascript, or draw the SVG.
There's an amusing story in The Social Network, when Mark Zuckerberg is scraping all the photos for his new website by what he calls "a little wget magic". In fact, wget is a command line tool that does exactly this — all he would need to do is wget --recursive (what our load function does, but following the links, too), with --domains to stay on one website, and --page-requisites to grab the necessary CSS and images, so as to download an entire website in a single command.
And the very first step in this process, DNS, which we already covered in the networking chapter:
>>> import socket
>>> sorted({info[4][0] for info in socket.getaddrinfo('www.python.org', 443, proto=socket.IPPROTO_TCP)})
['151.101.0.223', '151.101.128.223', '151.101.192.223', '151.101.64.223', '2a04:4e42:200::223', '2a04:4e42:400::223', '2a04:4e42:600::223', '2a04:4e42::223']
Several IPv4 and IPv6 addresses for one name; so python.org is actually hosted by a content delivery network, or CDN (that's where the Varnish headers from before came from): it serves the website from a myriad of different machines around the world, and it's the client that chooses which of these addresses to use. Remember this when we talk about caching and other layered systems.
Statelessness, Cookies and Sessions
Now, this request-response architecture is simple and powerful, but it has one big limitation: it's stateless. It doesn't remember anything; there's no conversation, no session. Every request must be self-contained and provide all the necessary information to be handled.
To understand what this means, let's consider an example. Suppose I don't keep state, and you want to ask me what my favourite book is — so you go ahead and ask. I tell you, it's The Unbearable Lightness of Being; and you say, OK, what is it about? But at this point, I have no idea what "it" refers to — so I have to ask, what what is about? Your question doesn't make sense without some context. In other words, you'd have to repeat yourself: regarding The Unbearable Lightness of Being, what is it about?
That's how web servers are: they don't keep state, so every request has to be as specific as possible, and include all the necessary information for the server to handle it. And in case you're wondering, yes: this is the book from which the title of this one is taken.
Anyway, back to the question: why would anyone design a protocol like this? On one hand, we talked about the complexities of stateful systems when we were discussing design; and on the other, it's a question of scalability. If I keep state, and you ask me the first question, and then ask someone else the second question — they wouldn't be able to answer it either. Not because they're stateless, but because there are two separate states. But if I don't keep state at all, then I can answer any question; so if there are a hundred of us, all identical, you can ask any of us anything, and a load balancer can distribute the load evenly. What's more, if one of us crashes, you can just go and ask someone else, right? Availability and scalability go hand-in-hand.
But this also means that something like a shared state, like a single database, can become a bottleneck — a single point of failure, really: if it's down, everyone is down; if it's loaded, everyone is slow. Sometimes, like with a database of users to check passwords against, it's inescapable; but in any case where it isn't, it's better to avoid it. One of my students asked me once how to implement a button that would add a black pixel to an image every time it's pressed — and the stateless solution I gave was to have the client say, "last time I saw your image, it had 17 black pixels," and get the image with 18 back.
But this is not the case with logins, is it? The whole point of the exercise is to get to some personalized and private content, so it would have to be stateful — the server would have to remember who you are on every subsequent request. To make this possible, there's a little hack in place, called cookies: a key-value pair that a web server can ask a web browser to store, per domain, in a Set-Cookie response header. The browser then attaches this to every subsequent request to that domain in a Cookie header, so the server gets this information back. This is known as carried state: every request "carries" all the context it has accrued with that server.
Let's walk through such a login flow with our local server, just like a web browser would, step by step.
>>> import requests
>>> response = requests.get('http://localhost:8000/account', allow_redirects=False)
127.0.0.1 - - [14/Oct/2026 19:47:46] "GET /account HTTP/1.1" 302 -
>>> response.status_code, response.headers['Location']
(302, '/login')
That's a 302 Found, with a Location header, telling us to go to /login instead. I'm passing allow_redirects=False here so we actually see it happen — otherwise, requests would follow the redirect automatically. Anyway, we go there and get:
>>> print(requests.get('http://localhost:8000/login').text)
127.0.0.1 - - [14/Oct/2026 19:47:46] "GET /login HTTP/1.1" 200 -
<!DOCTYPE html>
<html>
<body>
<form method="POST" action="/login">
<input name="username"> <input name="password" type="password">
<button type="submit">Log in</button>
</form>
</body>
</html>
This is a form with a method="POST" and an action="/login", so when you click the button, the browser sends a POST request to /login with the data from the input fields, urlencoded, in the request body (using their name attributes as keys). Let's do this:
>>> response = requests.post('http://localhost:8000/login', data={'username': 'alice', 'password': '1111'})
127.0.0.1 - - [14/Oct/2026 19:47:46] "POST /login HTTP/1.1" 403 -
>>> response.status_code, response.text
(403, '<p>Invalid username or password.</p>\n')
>>> response = requests.post('http://localhost:8000/login', data={'username': 'alice', 'password': '1234'}, allow_redirects=False)
127.0.0.1 - - [14/Oct/2026 19:47:47] "POST /login HTTP/1.1" 302 -
>>> response.status_code
302
>>> response.headers['Location'], response.headers['Set-Cookie']
('/account', 'session=ceae79a505514fe8; HttpOnly')
First, we try a wrong password, which results in a 403 Forbidden and a nice error page (some websites return a 200 with an error message instead); and then, we try the right one, which results in a 302, with a Set-Cookie header this time. What happens behind the scenes is that the server generates a random token, remembers that this token is associated with alice, and sends back a 302 to /account, along with that token — session=..., and HttpOnly means that Javascript on the page shouldn't be allowed to access it (so, if a script is injected by a hacker, it can't steal this cookie). What's more, the fact that we redirect after a POST means that if you refresh the page, the form won't be resubmitted (browsers follow a 302 after a POST with a GET, so a 303 See Other is more explicit in this case). Now, if we go to /account:
>>> token = response.cookies['session']
>>> requests.get('http://localhost:8000/account', headers={'Cookie': f'session={token}'}).text
127.0.0.1 - - [14/Oct/2026 19:47:47] "GET /account HTTP/1.1" 200 -
'<p>Welcome back, alice!</p>\n'
>>> requests.get('http://localhost:8000/account', headers={'Cookie': 'session=123'}, allow_redirects=False).status_code
127.0.0.1 - - [14/Oct/2026 19:47:47] "GET /account HTTP/1.1" 302 -
302
If we provide the right cookie, the server can look up its token, and use it to personalize the page; but if we provide a made-up cookie, it just bounces us back to the login page, like we never left. This is also why the token has to be unguessable (generated by secrets, not 123); and if someone does get a hold of it (like, if you send the request over plain HTTP and someone is tapping on the wire), they are you — they can simply provide this token to the server, and it will associate them with your account; one of the big reasons for HTTPS is to prevent this from happening.
If we were to do it with a requests.Session, by the way, it would look like this:
>>> session = requests.Session()
>>> response = session.post('http://localhost:8000/login', data={'username': 'bob', 'password': '5678'})
127.0.0.1 - - [14/Oct/2026 19:47:48] "POST /login HTTP/1.1" 302 -
127.0.0.1 - - [14/Oct/2026 19:47:48] "GET /account HTTP/1.1" 200 -
>>> [r.status_code for r in response.history], response.status_code, response.url
([302], 200, 'http://localhost:8000/account')
>>> response.text
'<p>Welcome back, bob!</p>\n'
>>> session.cookies.get_dict()
{'session': '657bf09f7337adda'}
A session manages a cookie jar and handles redirects; so in one call, we get two requests in the server log, and we can see the 302 we went through in history. This is exactly what your browser does all day long.
And if this solution sounds a bit patchy, that's because it is — it's pretty mindblowing that such a fundamental part of the web is so improvised. But it does achieve the desired goal of keeping the protocol itself stateless — it's the requests that carry the state, not the connection (although this server is still stateful, because it stores a table of sessions; other solutions, like JWTs, pack this information into the cookie as well, and use signatures to protect it from tampering, so any server can verify it without having to do a lookup).
APIs and REST
HTML, though, is for humans. If a program is trying to access that data — it has to "scrape" it off the HTML, parsing the document, digging out the cells from the rows from the table, and generally breaking when the website designer decides to move something around. It's not an API, really; compare this to json.loads, and getting dictionaries and lists you can iterate over directly.
And since HTTP is the lingua franca of systems, and everybody speaks it, kind of like everybody speaks English — it's pretty simple, and everybody knows it — it's worth asking why we can't just use it to transmit data in this simple and universal language.
And why do we even need APIs? Let's call this the compiler lesson. If there are $m$ different programming languages, and $n$ different processor architectures, we'd have to develop $m$ times $n$ different compilers to translate between them, right? Unless we can reduce the problem to a smaller one, like translating every language to some intermediate representation, and then translating that to every architecture — in which case, we only need $m$ plus $n$ different compilers. It's the same here: if we put our business logic behind some web API, we can develop our website, our mobile apps, our command-line interface and libraries for other languages all as thin clients of it. "All problems in computer science can be solved by another level of indirection," as David Wheeler put it.
So, what's the laziest way we can achieve this? If you're not afraid to get your hands dirty, you might come up with a single endpoint, which receives a POST request with some JSON data, describing which function we want to call and with what arguments, and returns a JSON response with the result. I call it lazy REST, although it's really remote procedure calls over HTTP — but it's pretty simple (about 20 lines of Flask), and we'll write it in the next chapter; here's a preview:
>>> import requests
>>> response = requests.post('http://localhost:8000/rpc', json={'function': 'add', 'args': [1, 2]})
>>> response.json()
{'result': 3, 'error': None}
>>> requests.post('http://localhost:8000/rpc', json={'function': 'div', 'args': [1, 0]}).json()
{'result': None, 'error': "KeyError('div')"}
Everything is 200, even when it doesn't work; that's an error message in the body, alright. And to be fair, it's simple, and it works — and for internal tools, it's often good enough. But it doesn't take into consideration anything that HTTP already gives us: URLs, verbs, status codes, headers and caching.
A proper REST API, then, is based on Roy Fielding's PhD dissertation from 2000 (and it's surprisingly not a terrible read). URLs, it suggests, are used for nouns: /users is a collection of users, /users/3 is a particular user. Then, verbs are used to describe the action we want to take: and here, we use the four standard ones known as CRUD: create, retrieve, update and delete — and it's pretty amazing how they cover 99% of what we do with data. To create, we use POST, either to the collection resource, in which case the server assigns an ID, or PUT to some specific URL, in which case the client requests it explicitly. To retrieve, we use GET, either on a collection resource to list all its items, or on a particular item to get it. To update, we use PATCH to change some fields, or PUT to overwrite the entire thing. To delete, we use DELETE. Then, there's some metadata in the headers, the body contains the data itself, and status codes are like exit codes — so:
>>> response = requests.get('http://localhost:8000/api/users')
>>> response.headers['Content-Type']
'application/json'
>>> response.json()
[{'id': 1, 'name': 'Alice Lovelace'}, {'id': 2, 'name': 'Bob Babbage'}]
And if we want to create a new user:
>>> response = requests.post('http://localhost:8000/api/users', json={'name': 'Null Pointress'})
>>> response.status_code, response.headers['Location']
(201, '/api/users/3')
>>> response.json()
{'id': 3, 'name': 'Null Pointress'}
>>> requests.get('http://localhost:8000/api/users/3').json()
{'id': 3, 'name': 'Null Pointress'}
Note that POST operates on a collection resource, and the server assigns an ID — so we get a 201 Created status code, and a Location header pointing to the newly created resource. We can also use POST to invoke an action that's not necessarily CRUD, like /users/7/wake-up.
To patch the user, then, and delete it, which happens with 204 No Content:
>>> requests.patch('http://localhost:8000/api/users/3', json={'name': 'Null Pointer'}).json()
{'id': 3, 'name': 'Null Pointer'}
>>> requests.delete('http://localhost:8000/api/users/3')
<Response [204]>
>>> requests.get('http://localhost:8000/api/users/3')
<Response [404]>
>>> requests.get('http://localhost:8000/api/users/3').json()
{'error': 'no user 3'}
We get 404 Not Found. And note that the body is still a JSON error message, because even API errors are for programs — it's just easier for us to debug if they're human-readable, too. And if we try to create a user with PUT (which creates the resource at the specified URL), and repeat it — and then try to delete the collection altogether:
>>> requests.put('http://localhost:8000/api/users/16', json={'name': 'Grace Hopper'})
<Response [201]>
>>> response = requests.put('http://localhost:8000/api/users/16', json={'name': 'Grace Hopper'})
>>> response.status_code, response.json()
(409, {'error': 'user 16 already exists'})
>>> response = requests.delete('http://localhost:8000/api/users')
>>> response, response.headers['Allow']
(<Response [405]>, 'GET, POST')
We get 201 Created, then 409 Conflict (although, strictly speaking, PUT is supposed to replace the resource, so repeating it should be harmless — different designs may vary); and for the collection, we get 405 Method Not Allowed, with a nice Allow header listing what methods are allowed. Designing URLs is not an exact science, though; the user's thoughts can be /users/7/thoughts, or /thoughts?user=7; the best advice I can give you, as is the case with any language, is to read as many other APIs as you can, and see how they phrase their resources. Also, don't be scared of REST pedants — they're like the Hebrew grammar pedants, and there is a right form, but nobody really speaks this way.
There are some other constraints in Fielding's dissertation, but they're pretty advanced: we already covered statelessness, and the others are cacheability (responses can specify how long they're valid for — that's the Cache-Control header, where the current time is good for a second, the 50th Fibonacci number is good forever, and only the server knows which is which — so caches, like your browser, your internet service provider or a content delivery network, can answer on its behalf, and that's the X-Cache: HIT we saw when we accessed python.org), a layered system (the client can't tell if it's talking to the server or an intermediary), and a uniform interface and HATEOAS: responses have links to related resources, so a client can navigate the API as a person would navigate hypertext — a language that can talk about itself. That last one is so cool, and so underused — almost nobody implements it.
Frontend, Backend, and a Murderous Evolution
Let's put the pieces together in a bit of a historical perspective, and consider four different eras. First, we had static websites: resources on a remote filesystem. Then, as people wanted to log in, have private content, and personalize the experience, the response had to be generated dynamically, per request and per user — think of Facebook baking your feed especially for you. So languages like PHP were invented, to combine HTML with code in an elegant enough way.
Then came web APIs: if HTTP is so convenient, why not use it to serve raw data to machines? I have a friend who has Xiaomi smart bulbs at home. When a bulb connects to his WiFi, it registers itself with Xiaomi's server, saying "hey, I'm here, I'm bulb 8, and I belong to so and so" — and when my friend logs into his app and presses a button, the app tells this server to turn on the bulb, and the bulb, which periodically polls the server to ask "what's my status?", eventually gets a "you're on" message and lights up. At least, that's what's supposed to happen. Services like IFTTT let you glue such APIs together, so when you say "remind me" to your speaker, it actually goes into your navigation app.
Finally, we have web applications: instead of baking HTML on the server per request, the server sends a small and simple page with some Javascript, but no data at all. This Javascript then makes a request to the API to fetch the data as JSON, and uses it to construct the page inside the browser. The result is the same as before: a website with some content. But this time, the same API that serves the website also serves the mobile app, the desktop app and any other scripts — so the complex logic of managing the data is solved once and for all.
So, if you'd ask someone who calls themselves a web programmer what exactly do they do, the answer would change over the years. First, "I'm really good at HTML, and I write static websites"; then, "I'm really good at PHP, and I write dynamic websites"; then, "I'm really good at the backend, and expose RESTful APIs"; and finally, "I'm really good at Javascript, and I write client-side applications".
The frontend is everything that runs on the client side: in the browser, or on the phone — the HTML, CSS and Javascript that the user actually sees. The backend is the server side: the databases, and the API that provides access to them. But notice how in this architecture, the frontend is just another client of the backend: it doesn't just display data, but actively requests it:
$ cat site/users.html
<!DOCTYPE html>
<html>
<head><title>Users</title></head>
<body>
<h1>Users</h1>
<ul id="users"></ul>
<script src="users.js"></script>
</body>
</html>
And in users.js:
$ cat site/users.js
async function loadUsers() {
const response = await fetch('/api/users');
const users = await response.json();
const list = document.getElementById('users');
for (const user of users) {
const item = document.createElement('li');
item.textContent = user.name;
list.appendChild(item);
}
}
loadUsers();
fetch is asynchronous, so it returns a promise, and async/await are pretty much the same as in Python (from the async chapter); response.json() parses the body; and the rest is the same DOM calls as before. This is also known as AJAX: asynchronous Javascript and XML — although it's almost always JSON these days. What a client that doesn't run Javascript gets is:
>>> import requests
>>> print(requests.get('http://localhost:8000/users.html').text)
127.0.0.1 - - [14/Oct/2026 19:47:55] "GET /users.html HTTP/1.1" 200 -
<!DOCTYPE html>
<html>
<head><title>Users</title></head>
<body>
<h1>Users</h1>
<ul id="users"></ul>
<script src="users.js"></script>
</body>
</html>
>>> requests.get('http://localhost:8000/users.js').headers['Content-Type']
127.0.0.1 - - [14/Oct/2026 19:47:55] "GET /users.js HTTP/1.1" 200 -
'text/javascript'
>>> requests.get('http://localhost:8000/api/users').json()
127.0.0.1 - - [14/Oct/2026 19:47:55] "GET /api/users HTTP/1.1" 200 -
[{'id': 1, 'name': 'Alice Lovelace'}, {'id': 2, 'name': 'Bob Babbage'}]
Requesting this page gives no users at all: they arrive in a separate request, by the script. But this is also why single-page applications feel so much like a desktop one: you click and the view changes without a refresh or a flicker, a spinner is shown while data is loading, and infinite scrolling is just a page that keeps growing by requesting more and more. The cost is a lot of Javascript that has to be downloaded and executed before anything becomes available, so rendering on the server has been making a comeback.
But if we think about it for a moment, it's actually a bit crazy. The web wasn't designed — it evolved. And it was a murderous evolution: technologies survived because they survived, not because someone made a decision; and as web programmers, we live in this messy, complex and beautiful world, and at any given moment have to know a bit of everything, because we can't know what exactly will survive. HTML is the one you can't get away from; CSS is the styling; and Javascript is the interactivity; and then there was jQuery, which was a flavor of Javascript that made it saner, and survived the test of time. Saying you write plain Javascript is a bit like saying you use Python with only import sys.
But if you think about it for another moment, it's even crazier — because underneath it all, there's what we saw in the first section: a process, running on a port, receiving text, and sending text in response. Everything we did so far, we did as the client: the browser, or curl, or requests. In the next chapter, we're going to switch sides: write this process ourselves, first with a raw socket, then with our own little framework, and finally with Flask.