Information Theory

01 · the missing quantity

Mutual information, and the tax for a wrong belief

Entropy measures the uncertainty inside one source. But the whole point of communication — and of evidence, and of perception — is that one thing tells you about another. The thunder tells you about the lightning; the test result tells you about the disease; the open goat door on Day 4 told you about the car. The quantity that captures this is the true workhorse of the theory: mutual information.

Start with two simpler pieces. Joint entropy $H(X,Y)$ is the total uncertainty in the pair. Conditional entropy $H(Y\mid X)$ is the uncertainty left in Y once you already know X — the surprise that survives. Mutual information is simply the gap between them: how much knowing X shrinks your uncertainty about Y.

For two variables, the overlap of two circles is a useful accounting picture. Do not push it too far: with three or more variables, information diagrams can contain signed interaction regions. Mutual information measures statistical dependence, not causation. A causal graph (Day 5) encodes structural assumptions and conditional independences; two variables can share information because one causes the other, because they share a cause, or because of selection. In Bayesian experimental design (Day 2), expected information gain about a hypothesis is one useful objective and can be written as a mutual information. A realized observation is evidence through likelihood ratios, Bayes factors, log scores, or posterior updates.

Mutual information measures statistical dependence. Channel capacity is the maximum mutual information rate achievable under a specified channel model and constraints. Causation and realized evidence require additional structure beyond mutual information alone.

The surprise tax: relative entropy

There’s a cousin of entropy that turns out to be one of the most useful quantities in all of science, and it answers a sharp question: what does it cost to believe the wrong thing? Suppose reality draws outcomes from a distribution $p$, but you’ve modeled the world as $q$. Every time you’re surprised, you’re surprised by the wrong amount — you budgeted $-{\log }_{2}q$ bits but reality charged you according to $p$. The average overpayment is the relative entropy, or Kullback–Leibler divergence:

This single inequality is doing astonishing amounts of work across the whole curriculum. It is the penalty for a bad prior — which is why it sits at the heart of Bayesian updating (Day 1, Day 4). KL is a divergence, not a true distance: it is asymmetric and lacks the triangle inequality. Cross-entropy is related but not identical: $H(p,q)=H(p)+D_{\mathrm{KL}}(p\Vert q)$, so minimizing cross-entropy is equivalent to minimizing KL only when the true distribution p is fixed. Language-model training can be idealized as minimizing an empirical estimate of expected cross-entropy, not as literally accessing $D(\text{reality}\Vert \text{model})$. Friston’s variational free energy is also subtler: it upper-bounds surprisal under a generative model and contains a KL term between an approximate posterior and the model posterior. Mutual information itself can be written as a KL divergence between the joint distribution P(X,Y) and the product P(X)P(Y).

02 · compression

Squeezing out the air

Entropy isn’t just an abstraction; it’s an asymptotic floor you can approach. Shannon’s source coding theorem — the quieter twin of the noisy-channel theorem — says that long blocks from a specified source can be compressed to an average rate arbitrarily close to H bits per symbol, and not below H without losing information or exploiting structure outside the source model. Entropy is the irreducible residue relative to that model: the message with the air squeezed out. Everything above that floor is redundancy, and redundancy is exactly what compression hunts.

How much air is in ordinary text? A great deal. With 27 symbols (26 letters and a space), a perfectly random stream would carry ${\log }_{2}27\approx 4.7$ bits per letter. But English is wildly predictable — q often drags u behind it, th and he recur, vowels are partly constrained. In a famous 1951 experiment, Shannon had people guess the next letter of a hidden text and measured how often they were right, arriving at an entropy of roughly one bit per letter (his estimates ranged about 0.6 to 1.3). The implication is startling: English is something like three-quarters redundant. Yu cn prbbly rd ths sntnc wth th vwls rmvd — because context already supplied much of the information needed to recover the vowels. Crossword puzzles, autocomplete, and the ability to hear someone at a noisy party all run on the same surplus.

All of that is lossless: unzip and you get back every bit. But your eyes and ears are forgiving instruments, and that forgiveness is worth money. Rate–distortion theory (Shannon, 1959) asks the harder question: if you’re allowed to lose a little, how few bits can you spend for a given level of acceptable distortion? This is the governing math of lossy compression — JPEG throwing away the fine color detail your retina can’t resolve, MP3 discarding the quiet tones a louder one is already masking. The art is to spend your bits only where a human will notice, and to let the rest evaporate. Streaming, video calls, the entire visual texture of the modern internet — all of it is rate–distortion theory cashed in.

03 · the ultimate compression

Kolmogorov complexity, where information meets computation

Shannon’s entropy is a property of a source — a probability distribution. But here’s a puzzle it can’t touch. Consider two strings of a million digits: one is a million random coin-derived digits, the other is the first million digits of $\pi$. To Shannon, sampled from “uniform digit sources,” they look equally dense with information. Yet $\pi$ can be regenerated from a comparatively short program, while a typical random string cannot be compressed much at all. The random-looking string has the higher description complexity.

That something is its Kolmogorov complexity (Solomonoff, Kolmogorov, and Chaitin, independently, in the 1960s): the length of the shortest computer program that outputs the string. It is the compression limit measured not against a probability model but against computation itself, and it is machine-independent only up to an additive constant determined by the chosen universal machine. $\pi$ is profoundly compressible (short program, infinite output); a typical long random string is incompressible in a precise counting sense — most strings have no description much shorter than themselves. In fact this becomes a definition of algorithmic randomness.

And here the subject grows a sharp, beautiful tooth. Kolmogorov complexity is provably uncomputable proven. There is no algorithm that, given any string, returns the length of its shortest program — a result that rhymes with the Gödel and Turing limits waiting on Days 27–28, and with Chaitin’s eerie constant Ω, a specific, perfectly well-defined number. A suitable consistent formal system can determine some finite facts about Ω, but not arbitrarily many of its digits. We will meet related ideas again tomorrow under the heading of complexity, but with an important warning: high description complexity alone does not measure meaningful or organized complexity. Typical random strings maximize it too.

04 · codes against the dark

How information survives a hostile world

The main lesson promised that, below a specified channel’s capacity, vanishing error is asymptotically possible if you can find sufficiently long codes. That treasure hunt deserves its own telling, and it starts with a frustrated man at Bell Labs on a Friday night.

In 1947, Richard Hamming had weekend access to a relay computer, and it kept letting him down: whenever it detected an error in his input, it simply halted and waited for a human who wouldn’t arrive until Monday. “If the machine can detect an error,” Hamming fumed, “why can’t it locate and correct it?” Out of that irritation came the Hamming code (published 1950), the first true error-correcting code, and with it a geometric way of seeing the whole field.

The idea is Hamming distance: the number of bit-flips separating two codewords. If your only valid messages are 000 and 111, they sit distance 3 apart. A single error turns 000 into something like 010 — but 010 is still closer to 000 than to 111, so the receiver can quietly correct it by majority vote. Spread your valid codewords far enough apart in this space of bit-strings, and you carve out a little protective “sphere” of correctable errors around each one. Redundancy, deployed with geometric cunning.

From Mars to your pocket

Hamming’s geometry scaled into an engineering miracle that you rely on hourly without noticing. Reed–Solomon codes (1960) treat data as points on polynomials and can repair whole bursts of damage — which is why a scratched CD still plays, why a torn QR code still scans, and why the Voyager probes could whisper pictures of the outer planets across billions of kilometers on a transmitter weaker than a refrigerator bulb. Deep-space missions stacked codes inside codes (concatenation) to claw their signal out of almost pure noise.

But for decades a gap remained between what Shannon promised and what engineers could build. That gap closed in a rush. Turbo codes (1993) stunned the field by getting within a fraction of a decibel of capacity using two simple coders trading guesses back and forth. Low-density parity-check codes — invented by Robert Gallager in 1960, then forgotten for thirty years as too demanding for the hardware of the day, and rediscovered in the late 1990s — now armor Wi-Fi, 5G data channels, and storage. And in 2009, Erdal Arıkan’s polar codes became the first explicitly constructed family proved to achieve the symmetric capacity of binary-input memoryless channels asymptotically, with efficient encoding and decoding established. In 2016 the global 5G standard adopted polar coding for control-related channels, while LDPC carries much of the data-channel burden. Sixty-eight years after Shannon drew the X on the map, engineers had practical paths toward it.

05 · the shoulders & the showman

The people behind the page

Shannon didn’t arrive from nowhere. Two Bell Labs predecessors had circled the idea. Harry Nyquist (1924) worked out how fast a telegraph line could signal; Ralph Hartley (1928), in a paper actually titled “Transmission of Information,” proposed measuring information as the logarithm of the number of possible messages — the seed of the bit, missing only Shannon’s crucial leap of bringing in probability and throwing out meaning. Shannon honored the debt: the unit of information in some conventions is still the “hartley.”

But the man himself is one of science’s great delights, and worth a paragraph of pure pleasure. Claude Shannon rode a unicycle down the Bell Labs corridors while juggling. He built a mechanical mouse named Theseus (1950) that could learn its way through a maze — one of the first demonstrations of machine learning, sitting on his desk. He built, at Marvin Minsky’s suggestion, the “Ultimate Machine”: a box with a single switch that, when flipped on, opened, extended a mechanical hand, switched itself off, and withdrew. With the mathematician Ed Thorp he built what is often called the first wearable computer — a cigarette-pack-sized device to beat roulette, smuggled into Las Vegas in 1961. His parlor trick of having guests predict the next letter of a sentence wasn’t a party game; it was him measuring the entropy of English in real time. The inventor of the most disembodied, abstract quantity in all of engineering was, in person, irrepressibly, gloriously physical. It is fitting that the punchline of his theory — information is physical — was waiting at the end of the road he started down.

06 · when information goes quantum

The bit’s stranger sibling

Everything so far assumed a bit is a definite 0 or 1. Quantum mechanics offers a richer carrier — the qubit — and the information theory built on it is so different it deserves its own block (Day 47). Here is just enough to feel the strangeness.

A qubit can hold a superposition of 0 and 1, a point on a whole sphere of possibilities rather than two poles. You might think it therefore carries infinitely more information. It doesn’t — and the reason is one of the deepest facts in the subject. Holevo’s bound (1973) limits the accessible classical information from a quantum ensemble; in the ordinary unassisted setting, one transmitted qubit cannot reveal an arbitrary continuum of classical data. Entanglement-assisted protocols such as superdense coding change the resources being counted. Quantum measurement can also have more than two outcomes, so “yes/no” is a simplification. The continuum is real but coy: it shapes how the qubit evolves and interferes, yet cannot be fully read out as ordinary classical information.

Two more jewels, both forward-pointers. The no-cloning theorem (Wootters & Zurek; Dieks, 1982) established says an unknown quantum state cannot be copied — there is no quantum photocopier — which helps explain why eavesdropping can disturb quantum-cryptographic protocols, though security requires a full protocol proof, not no-cloning alone. And the quantum analogue of Shannon entropy, the von Neumann entropy $S=-\mathrm{Tr}(\rho \log \rho )$, measures uncertainty in a quantum state and equals entanglement entropy for a pure bipartite state. Mixed-state entanglement needs other measures. The bit, it turns out, was only the first draft.

07 · cheating the toll

Beating Landauer with reversible computing

The main lesson delivered Landauer’s standard verdict: resetting an initially unknown fair bit in an isothermal cycle costs at least $k_{B}T\ln 2$ because the operation crushes two possible pasts into one. But read the fine print and a loophole gleams. The fundamental cost is charged on logically irreversible steps — the ones you can’t run backward. What if you simply never erased anything?

This is reversible computing, and it is theoretically sound established in principle. Charles Bennett showed in 1973 that any computation can be rebuilt as a sequence of reversible steps that never destroy information — instead of overwriting intermediate results, you keep them, run your computation, copy out the answer, then run the whole thing backwards to cleanly “uncompute” the garbage, returning every bit to its start with no net erasure. Special reversible logic gates make this concrete: the Toffoli gate (a controlled-controlled-NOT) and the Fredkin gate (a controlled swap) are universal — you can build any circuit from them — yet each is perfectly invertible, with the same number of outputs as inputs and nothing thrown away.

In principle, then, reversible computing can avoid the fundamental erasure toll until information is intentionally discarded. Physical implementations can reduce dissipation by using gentler, more nearly quasistatic protocols, but the speed-energy trade-off depends on the device model; logical reversibility by itself does not mean a computer must run vanishingly slowly. Reversible algorithms also involve time-space trade-offs rather than simply storing an unlimited history forever. No one has built a useful general-purpose reversible computer; it remains an exquisite idea more than a product. But it is exactly why the field is suddenly interesting again: as conventional chips press toward physical limits, not forgetting starts to look like an engineering strategy rather than a curiosity. The thread running into Day 47: ideal closed-system quantum gates are unitary and reversible, while initialization, measurement, error correction, control, and reset require their own thermodynamic accounting.

08 · the last wall

The ultimate physical limits of computation

Landauer bounds the energy of forgetting. But there are other walls — on speed, on density, on how much computing a lump of matter can ever do — and tracing them out leads somewhere genuinely vertiginous. In 2000, MIT’s Seth Lloyd estimated an “ultimate laptop”: one kilogram of matter, one liter of volume, organized as efficiently as the laws of physics permit (Nature, 31 August 2000). These are model-dependent upper bounds, not a practical device specification.

The speed limit comes from quantum mechanics itself. The Margolus–Levitin theorem (1998) bounds how fast a system can pass through distinguishable states in terms of energy above its ground state, with the exact interpretation depending on what is counted as an operation. Pour all of one kilogram’s rest-mass energy ($E=m{c}^2$, about $10^{17}$ joules) into Lloyd’s idealized computation, and you get a ceiling of roughly $10^{51}$ operations per second. The memory limit comes from counting the states such a system can occupy: about $10^{31}$ bits. These numbers are not engineering targets; they are made of $c$, $\hslash$, and $k_B$, the bedrock constants. Lloyd notes the obvious snag — a device running at this limit has turned its kilogram into a roughly billion-degree plasma, “a packaging problem,” he dryly concedes.

Push the same accounting to the largest scale and you can ask: how much has the entire universe computed since the Big Bang? Lloyd’s 2002 estimate is about $10^{120}$ elementary operations on $10^{90}$ ordinary-matter bits, with a much larger information budget if gravitational degrees of freedom are included. Enormous, yes. But finite. There is a hard, countable ceiling on how much computation reality itself has performed. Whether that means the universe quite literally is a computer is a live and contested interpretation contested — interpretation, not result, a close relative of the “it from bit” overreach the main lesson flagged. The numerical bounds are physics; the metaphysics is optional.

09 · the thread, everywhere

Once you have the unit, you see it everywhere

The quiet power of Shannon’s idea is that the moment information became a number, it stopped belonging to telegraphy and started turning up in every field that deals with uncertainty, variety, or surprise. A quick tour of the thread’s reach, each a doorway to a later day:

• In ecology, the Shannon diversity index — the very same H = −Σ p log p — measures biodiversity: a rainforest has high entropy (many species, evenly spread), a monoculture field has almost none. Surprise, applied to “which species is this?”
• In machine learning, autoregressive language-model pretraining commonly minimizes empirical token-level cross-entropy. Perplexity is the exponential of that average loss and measures predictive fit on a specified dataset, but it is not a complete measure of model quality. Instruction tuning, preference optimization, reinforcement learning, distillation, and multimodal training can use other objectives.
• In biology, DNA is a four-letter code carrying ~2 bits per base — heredity as literal digital information storage, a framing that runs straight into the central dogma on Day 77 and the origin of life on Days 87–90.
• In physics, the entropy we defined today and the thermodynamic entropy of heat engines turn out to be — perhaps — the same thing, the question the main lesson flagged and that Day 33 will reopen as the arrow of time.
• In life itself, Schrödinger’s idea that organisms survive by feeding on “negative entropy” (Brillouin later called it negentropy) is information thermodynamics in embryo — the puzzle that Days 83–85 are built around, where today’s $k_{B}T\ln 2$ returns to govern the living cell.

That is the real reason this day sits where it does in the course, only seventh of a hundred and eighty. Information is one of the five threads — alongside energy, evolution, emergence, and computation — precisely because, once weighed, it refuses to stay in its lane. It is the connective tissue of the whole map.

01 · the headline result

Measuring irreversibility inside a living cell

If we had to name one recent result that could be remembered in twenty years, it is this: in 2024, a team led by Felix Ritort at the University of Barcelona published a way to infer the rate at which a living cell produces entropy — the quantitative arrow of time inside a single red blood cell — from microscopy plus force calibration and model assumptions.

The insight is a new mathematical identity called the Variance Sum Rule. It connects fluctuation statistics of a probe with the restoring forces acting on it, allowing entropy production rate $\sigma$ to be estimated under the model’s conditions. No calorimeter, no complex perturbation protocol — but not just a bare microscope either.

They pointed this tool at human red blood cells and mapped the entropy production rate across the membrane. What they found was striking: $\sigma$ is spatially heterogeneous, varying from spot to spot across the cell, with a finite correlation length of about 0.6 micrometres. The average agreed with independent calorimetry measurements. The authors describe this as the first heat map of entropy production in a living system — a major step from qualitative “far from equilibrium” language toward spatially resolved thermodynamic accounting. established under model assumptions

Two companion results deserve note. Skinner & Dunkel (PNAS, 2021) developed a rigorous optimization framework for bounding entropy production from partial observations and applied it to bacterial flagellar motors and growing microtubules. And a 2024 PNAS paper used deep-learning probability flows — score-based generative models — to estimate entropy production in high-dimensional active-matter systems, offering a computational path where the analytic rule runs out of dimensions. both established

02 · the new rules

Universal trade-offs: uncertainty, speed, and cost

The deepest conceptual advance in stochastic thermodynamics since 2020 is a family of results that say: precision often costs entropy, under stated dynamical assumptions. If a molecular motor turns with low noise, if a clock ticks with reliable rhythm, if a sensor measures a concentration accurately — something may have to pay, and the currency is dissipation. These thermodynamic uncertainty relations (TURs) are powerful constraints for broad classes of nonequilibrium systems, but they are not assumption-free laws of nature.

Taken together, these results are building something like a thermodynamic code — a body of constraints on specified nonequilibrium processes, just as Shannon’s channel capacity constrains a specified communication model. The analogy is deliberate: the people doing this work view Shannon-era information theory and 21st-century stochastic thermodynamics as two chapters of the same subject.

03 · the cost of a tick

Clocks run on entropy

Here is a fact that, once you hear it, feels obvious: in important clock models and experiments, higher accuracy comes with higher dissipation.

The team — Pearson, Guryanova, Erker, Laird, Briggs, Huber & Ares, spanning Oxford, Vienna, and Lancaster — built the simplest possible clock: a nanometre-thick mechanical membrane vibrating in a cryogenic cavity, its oscillations read out by a radio-frequency circuit. By varying the drive power, they tuned the clock from sloppy to sharp and measured both accuracy and entropy production. They found a linear relationship: double the accuracy, double the dissipation. Their clock operated within an order of magnitude of the theoretical minimum. established

Follow-up work has deepened this. Meier, Schwarzhans, Erker & Huber (PRL 131, 220201, 2023) showed the cost splits into accuracy (regularity) and resolution (tick rate), both requiring entropy in their framework but in different ways. And a 2025 PRL paper on the entropic costs of extracting classical ticks from a quantum clock argued that the dominant cost can be not the clockwork but the measurement — the act of amplifying a quantum oscillation into a classical signal. The demon’s cousin again: the cost is not in the knowing, but in the making-classically-known.

04 · the highest-variance bet

Thermodynamic computing: noise as a feature

For seventy years, engineering’s response to thermal noise has been: suppress it. Every transistor is designed to overpower the random jitter of its atoms with a clean voltage swing many times larger. That swing is where most of the energy goes and most of the heat comes from. The thermodynamic-computing programme asks: what if you stopped fighting the noise and started computing with it?

The idea is surprisingly direct. A system of coupled physical oscillators — RLC circuits, resistor networks — will, if left alone, thermally fluctuate. Those fluctuations sample from the Boltzmann distribution of the system’s energy landscape. Design the landscape carefully, and those samples solve problems: linear systems, Bayesian inference, energy-based generative models. The computation is not imposed against the physics; it is the physics.

05 · demons, upgraded

Information engines go many-body, quantum, and autonomous

The single-particle engines of 2010–2021 were stunning proofs of concept. The recent push is in three directions: scale them up, make them quantum, and embody more of the sensing and feedback inside the device.

Many-body engines

Chor, Sohachi, Rosen, Rahav & Roichman (Phys. Rev. Research 5, 043193, December 2023) extended the Szilárd engine from a single particle to a many-body colloidal suspension, extracting work from collective number fluctuations. A follow-up (arXiv:2512.01942, December 2025, preprint) describes a “piston-like information engine” that harvests work from an equilibrium bath purely by conditional measurement — Maxwell’s demon writ in a crowd. PRResearch paper established piston preprint pending

A quantum engine charging a quantum battery

Zhang et al. (Kim group, Tsinghua; Phys. Rev. Lett. 135, 140403, 2025) built a cyclic quantum information engine from a single ytterbium-171 ion, using rapid mid-circuit measurement to suppress measurement disturbance. The energy extracted per cycle was used to charge a quantum battery. They report an information-to-ergotropy conversion efficiency of 67% of the theoretical maximum and information-to-work efficiency of 70%. established

Autonomous demons

Autonomous Maxwell demons have been demonstrated experimentally since at least the 2015 single-electron work of Koski and colleagues. In such systems, measurement and feedback are implemented internally rather than by an externally timed controller. That does not mean the device uses no nonequilibrium resources or produces no entropy: the energetic and entropic resources still have to be included in the full accounting. Newer quantum-dot rectifiers (PRR 2019), stochastic-resetting demons (PRR 2023), and transistor-based demons (PRB 2025) extend that design pattern for nanoscale energy harvesting. promising — individual realizations established

A real molecular motor as a demon

Amano, Esposito, Kreidt, Leigh, Penocchio & Roberts (Nature Chemistry 14, 530, 2022) performed a rigorous information-thermodynamic analysis of a real chemically-driven synthetic rotary molecular motor from the synthetic molecular-machines tradition recognized by the 2016 Nobel Prize in Chemistry, showing its operation decomposes into an information-processing cycle. The demon, it turns out, was always just a particularly well-designed molecule. established

06 · erasure goes quantum

Cooperative erasure and beyond-Landauer protocols

Buffoni & Campisi (Quantum 7, 961, 2023) erased 256 qubits simultaneously on a D-Wave Advantage quantum annealer by exploiting spontaneous symmetry breaking amplified by quantum tunnelling. The collective flip achieved a per-bit erasure cost approaching $k_{B}T\ln 2$ with ~99.9% success rate, while the per-bit action (energy × time) reached $10^{-22}$ erg·s — extraordinarily fast and efficient. The key: cooperative effects let the system cross the energy barrier collectively rather than bit by bit. established

A separate theoretical programme explores ancilla-assisted erasure (arXiv:2402.15812, 2024), in which an auxiliary quantum system shuttles entropy away from the memory, releasing less than $k_{B}T\ln 2$ of heat to the local bath in specific operating regimes. The authors stress the second law is not violated — the total entropy, including the ancilla, still increases — but the local heat can be reduced below the textbook floor. This is a pointer toward ultra-low-dissipation quantum computing, not a free lunch. promising — regime-specific, preprint

07 · information at the edge of spacetime

The island formula and the Page curve

The longest-running open problem in quantum gravity is the black-hole information paradox: does the information a black hole swallows survive its evaporation? Hawking’s 1975 calculation said destroyed; quantum mechanics says nothing is permanently lost. For forty-five years, nobody had a satisfying resolution.

In 2019–2020, a series of papers changed the conversation. Penington (JHEP, 2020), Almheiri, Engelhardt, Marolf & Maxfield (JHEP, 2019), and Almheiri, Hartman, Maldacena, Shaghoulian & Tajdini (JHEP, 2020) found that if you compute the entropy of Hawking radiation using a gravitational path-integral method — replica wormholes — the answer includes contributions from disconnected regions of spacetime called “islands” inside the black hole’s interior. When these kick in, the radiation’s entropy turns over and begins to decrease, tracing the Page curve — exactly the shape a unitary (information-preserving) theory predicts in those controlled models.

Island calculations are a leading theoretical resolution in controlled semiclassical-gravity models. They strongly support unitary evaporation there; they are not yet a complete microscopic account of realistic black holes.

The authoritative review is Almheiri, Hartman, Maldacena, Shaghoulian & Tajdini, “The entropy of Hawking radiation,” Reviews of Modern Physics 93, 035002 (2021). established as a theoretical result in model settings

The caveat: there is no experimental or observational test. The calculation lives inside a specific theoretical framework, and it is not clear it applies to the real, four-dimensional, non-AdS universe. The result is as mathematically secure as anything in the field — but the field itself floats above experiment. no experimental test

08 · the big picture

A unified physics of computation?

The most ambitious attempt to pull this together is a 2024 PNAS perspective by Wolpert, Korbel, Lynn, Tasnim, Grochow, Kardeş and seventeen co-authors: “Is stochastic thermodynamics the key to understanding the energy costs of computation?” Their argument: the thermodynamics of computation is not just about Landauer’s limit — that’s merely the floor. Above it lies a rich structure of mismatch costs, communication costs, and nonequilibrium constraints that form a comprehensive physics of computation as rigorous as Shannon’s theory of communication.

One comparison from their review stops you in your tracks: biological systems — the molecular machinery of a cell — can appear roughly 10⁵ times more energy-efficient per operation than some human-built computers, depending on what is counted as an operation and a boundary. Treat that as an illustrative, definition-dependent gap, not a universal constant. The main lesson’s Landauer-ladder diagram showed chips sitting far above the thermodynamic floor; the deeper point is that biology often computes closer to physical noise and dissipation limits than conventional digital engineering, and understanding how is a foundational open problem. established framework, illustrative ratio