Models, Maps & Idealization

Room Ithe long prehistory

Saving the phenomena: an argument four centuries deep

The main page framed realism versus instrumentalism as a twentieth-century affair: Putnam, Laudan, van Fraassen. But the fight is ancient, and for most of its life the instrumentalists were winning. The tradition even has a battle cry, handed down from Greek astronomy: sōzein ta phainomena, “to save the phenomena.” The job of a model, on this old view, was never to say what the heavens are, only to reproduce what they appear to do, to get the dots in the right places on the right nights.

Ptolemy’s epicycles, wheels turning on wheels to explain the planets’ loop-the-loops, were the original useful fiction. Many of his readers, and most medieval astronomers, treated them frankly as calculating devices. Nobody had to believe the sky was full of literal gears. The gears just had to work. That is instrumentalism in its cradle, a thousand years before the word existed.

The drama sharpened with Copernicus. When On the Revolutions of the Heavenly Spheres went to press in 1543, the Lutheran theologian Andreas Osiander slipped an unsigned preface into the front, without the dying Copernicus’s blessing, assuring readers that the wild new sun-centered scheme need not be true, only a convenient hypothesis for computing positions. It was a peace treaty with the authorities, written in pure instrumentalist ink: relax, it is just a better calculator.

By the early 1900s the physicist-philosopher Pierre Duhem, whom we met on Day 2 for the “you can always blame an auxiliary” thesis, had turned this into a full doctrine. A physical theory, Duhem argued in The Aim and Structure of Physical Theory (1906), is not an explanation of hidden reality but an economical classification of experimental laws: a supremely compressed filing system for the phenomena. Truth about unseen causes was not the goal. Tidy, predictive bookkeeping was. Three hundred years of “save the phenomena,” distilled. The modern instrumentalist owes Duhem the blueprint.

Room IIrealism strikes back

The ether’s one perfect prophecy

The main page used the luminiferous ether as Exhibit A in Laudan’s graveyard, a wildly successful theory whose central object turned out not to exist. But the ether’s case is far more disturbing than a simple “it worked, then it died,” and the reason is a single beautiful episode that the realist keeps in her back pocket.

Around 1818, the French Academy ran a prize competition on the nature of light. Augustin Fresnel submitted his wave theory: light as vibrations in the ether. On the judging panel sat Siméon Denis Poisson, a committed believer in the rival particle theory, who did some math with Fresnel’s equations and triumphantly derived an apparently absurd consequence. If Fresnel were right, the shadow of a small round disc should have a bright spot dead in its center, where the waves bending around every edge arrive back in step. Ridiculous, said Poisson: a clear reductio that sinks the whole wave theory. Another judge, François Arago, went to the bench and actually did the experiment. The spot was there.

This is the realist’s haymaker, and it has a name: a novel prediction. The ether theory did not merely accommodate facts already known. Anyone can curve-fit the past. It predicted something nobody had ever seen and no one expected, a detail so specific and strange that its coming true looks like no coincidence at all. This is the no-miracles argument from the main page at full strength: surely a theory cannot pull a rabbit like that out of a hat unless it has hold of something real. And yet there is no ether. The single best evidence realism ever assembled was produced by a theory we now call false. The graveyard, it turns out, contains a genius.

Divide and conquer

So how does the realist survive her own best example? With a move sometimes called selective, or divide-and-conquer, realism, sharpened by Philip Kitcher and Stathis Psillos in the 1990s. The trick is to stop being a realist about whole theories and become one about parts. In any successful theory, only some posits actually do the predictive heavy lifting: the working posits. Others are idle, along for the ride. Be a realist only about the working parts.

Run the ether through this filter and watch it sort itself. What did the real work in deriving Poisson’s spot? Not the substance “ether,” not its density, its elasticity, or the question of what it was made of. The work was done by the mathematical form of the wave equations: the relationships, the structure. And those carried over into Maxwell’s field theory and beyond. The idle metaphysical filler, the stuff of the ether, is exactly what got buried. So the pessimistic induction loses some of its teeth. Science did not keep the furniture and then throw it out; it kept the structure, which survived revolution, and quietly dropped the decoration. This is precisely the structural realism the main page ended on, now shown doing real work against its hardest case.

Room IIIthe callback to Day 4

Is the whole debate a base-rate fallacy?

Here is the room that should make a Day 4 graduate sit up. Recall the base-rate fallacy: a test that is 99% accurate for a disease that afflicts 1 in 10,000 still flags far more healthy people than sick ones, because you forgot to ask how common the disease is to begin with. Confusing P(positive | sick) with P(sick | positive) is the classic trap.

In 2004, the philosophers P. D. Magnus and Craig Callender pointed out that both sides of the realism war may be walking straight into it. Look again at the no-miracles argument. It wants to conclude that a successful theory is probably true, that is, that P(theory true | theory successful) is high. But by Bayes’ theorem, you cannot get there from P(success | truth) alone. You also need the base rate: the prior proportion of true theories in the pool of candidate theories science draws from. And that number is not just unknown. Magnus and Callender argue that it is unknowable, because there is no fair way to count or sample all candidate theories that could have existed for mature sciences. No base rate, no inference.

The same blade cuts the pessimistic induction, which tries to push P(theory true | success) low from the historical record of failures, and trips over the same missing denominator. Magnus and Callender’s diagnosis is almost clinical: framed probabilistically, the grand “wholesale” arguments are not just unresolved; they may be malformed, each a base-rate fallacy in a frock coat. Colin Howson made a closely related charge against no-miracles in 2000. The Day 4 machinery does not just describe this dispute. It may quietly dissolve it. live dispute

Room IVthe grammar of useful lies

Idealization has rules, and a famous tradeoff

The main page treated “deliberately false assumptions” as one thing. The literature is fussier, and the distinctions matter. First, separate two moves people lump together. Abstraction is leaving something out: modeling a planet’s orbit without mentioning its color. True as far as it goes, just silent. Idealization is actively misrepresenting: declaring the plane frictionless when you know full well it is not. The frictionless plane is not a gap in the description. It is a fib. The two feel similar and behave very differently.

Then there are kinds of idealization, and the third one is the one the main page skipped. There is Galilean idealization, which distorts for tractability and promises to add detail back later, and minimalist idealization, the spherical cow strategy that keeps only the causal factors thought to matter. But there is also multiple-models idealization: building several mutually incompatible models of the same thing, each false in a different direction, and triangulating among them. No single map is trusted. The agreement between differently wrong maps is.

“Our truth is the intersection of independent lies.” Richard Levins, “The Strategy of Model Building in Population Biology,” 1966.

Levins’s point, now called robustness analysis, is that if you attack a problem with several models built on different simplifying falsehoods, and they all spit out the same qualitative result, then that result probably depends on the real shared structure of the problem rather than on any one model’s particular lie. The lies are independent. Their intersection is where truth is most likely hiding.

Levins is also the source of modeling’s most famous impossibility claim: you cannot maximize generality, realism, and precision all at once. Push any two as far as they will go and the third has to give. The dial on the main page measured fidelity against usefulness. Levins’s triangle is the same wisdom in three dimensions: pick your sacrifice.

Room Vwhat is a model, really?

Mediators, fictions, and Maxwell’s imaginary gears

Step back and ask the question the main page tiptoed around: when a scientist “has a model,” what kind of thing is it? Three answers illuminate different corners.

A theory is a family of models. On the older syntactic picture, a scientific theory was a big set of sentences: axioms and their logical consequences. The modern semantic view says no. A theory is better understood as a collection of abstract structures, together with a claim that some real system is similar to one of them in specified respects. Newtonian mechanics is not fundamentally a wall of equations. It is a toolkit of idealized systems, the pendulum, the two-body orbit, the harmonic oscillator, plus the bet that bits of the world resemble them closely enough. Ronald Giere called the resulting stance perspectival realism: our models are like instruments, each giving a partial, perspective-bound view, true to the world the way a map is true, never the way a mirror is.

Models are mediators. In the influential 1999 collection Models as Mediators, Mary Morgan and Margaret Morrison argued that models are not simply read off from theory, nor simply summaries of data. They sit between, partly autonomous from both. You build a model the way you build an instrument, making pragmatic choices theory does not dictate; then, crucially, you learn by manipulating it, the way you learn by turning the knobs on an apparatus. A model is a tool you think with, not just a picture you look at. This is also why simulation, in the next room, can feel so much like experiment.

Models are fictions. The most provocative view takes the frictionless plane at its word: it is a fiction, an object that does not exist, described as if it did. The idea is old. Hans Vaihinger’s 1911 The Philosophy of “As If” argued that whole swathes of science and mathematics run on useful falsehoods we knowingly treat as if true. Recall the Day 3 surprise that Sherlock Holmes’s “deductions” are really abductions: here is its cousin. A scientific model may be closer to a character in a novel than to a photograph. The ideal gas is a bit like Sherlock: not real, never was, but reasoning about what it would do delivers real understanding.

Cartwright’s dappled world

One more turn of the screw deepens the entity realism from the main page. Nancy Cartwright, who gave us “the laws of physics lie,” built that thought into a whole worldview in The Dappled World (1999). Her claim: fundamental laws are true only inside models, and they describe the real world only where we have painstakingly engineered, or luckily found, a nomological machine: a shielded, stable, repeating arrangement of parts that produces lawlike regularity. A physics lab is such a machine. So is the solar system, roughly.

But most of the world is not shielded or stable, and there the tidy laws simply do not reach. They hold only ceteris paribus, all else equal, and all else is never equal. The world, for Cartwright, is not a pyramid with grand laws at the base but a patchwork: a dappled quilt of little domains where different models work and between which nothing universal runs. Read her way, “all models are wrong” stops being a remark about our maps and becomes a claim about reality: the laws themselves were idealizations all along. live position

Room VIthe trust problem

Verifying the unsurveyable, and a bombshell from 1994

Engineers who stake lives on simulations drew a hard distinction long ago. Verification asks: did we solve the equations right? Is the code a faithful numerical solution of the model we wrote down? Validation asks the harder thing: did we solve the right equations? Does the model actually correspond to the world? Solve-it-right versus solve-the-right-thing. Almost every dispute about trusting a model is really a dispute about validation.

And then, in 1994, three researchers detonated a small bomb under the word itself. In Science, Naomi Oreskes, Kristin Shrader-Frechette, and Kenneth Belitz argued that for models of open natural systems, climate, groundwater, ecosystems, verification and validation are impossible in principle. Their reasoning lands on three course threads at once. Natural systems are never closed, so you can never control all the inputs. Model solutions are non-unique, so a good fit never proves your model is the right one, which is the underdetermination from the main page. And confirming a model by matching its predictions to data is, strictly, the fallacy of affirming the consequent from Day 3: if my model is right, I will see X; I see X; therefore my model is right. Their verdict on the word “validation” is severe: it falsely implies a legitimacy the model cannot earn. The primary value of models, they argued, is heuristic. They are instruments for thinking, not certificates of truth. established

So how do we trust the black box?

This is where the main page’s neural weather model comes back to haunt us. If you cannot validate a model of an open system, and you cannot even survey the billions of weights inside a learned emulator, Humphreys’s epistemic opacity from the main page, then on what grounds could you ever trust its forecast?

Juan Durán and Nico Formanek proposed one answer in 2018: computational reliabilism. Stop and feel the click. Reliabilism was Day 1: the idea that a belief is justified not because you can recite an argument for it from the inside, but because it was produced by a reliable process, such as good vision or sound memory. Durán and Formanek point that externalist answer at simulations. You do not need to see inside the opaque model to trust its output; you need evidence that the process producing it is reliable.

That evidence looks like a track record of verification and validation where checking is possible, robustness across independent methods, a history of the technique succeeding where it could be checked, and the judgment of experts who know its failure modes. Trust migrates from transparency to reliability. The black box can be known the way a witness can be known: not by cracking open the skull, but by examining the record. The thing that defined a justified belief in a human brain on Day 1 turns out to define a trustworthy answer from a machine on Day 10. debated

Bonus room

Where “digital twin” actually came from

The main page leaned on the 2024 National Academies definition: bidirectional, continually updated, predictive. Where did that idea come from, and what makes the strict definition matter?

The ancestor flew on Apollo. NASA built physical twins of its spacecraft and kept them on the ground. When Apollo 13’s oxygen tank ruptured 200,000 miles from home, engineers fed live telemetry into ground-based simulators and physical mockups to improvise fixes for a craft they could not touch. A model on Earth, updated with data from its endangered counterpart in space, used to make a life-or-death decision: that is the platonic form of a digital twin, decades before the phrase existed.

The modern concept was laid out by Michael Grieves in 2002, in a University of Michigan product-lifecycle lecture: a slide already showing real space, virtual space, and data flowing both ways between them. The catchy name came later. NASA’s John Vickers coined “Digital Twin” around 2010, and it entered NASA’s technology roadmap. From about 2014, Siemens, Dassault, GE, ANSYS, and the rest stamped it across their marketing, which is exactly why a sober definition became necessary, and why the National Academies drew the line where they did.

That line is best seen as a three-rung ladder, following Kritzinger and colleagues (2018): what earns the name “twin” is not how detailed the model is, but how the data flows.

The taxonomy is deliberately austere. A plain digital model can be enormous and beautiful; a true twin can be visually boring. The difference is the loop.

Bonus room

A model of a model of a model

One last vista reframes the whole frontier. The main page set up a contrast between clean physics-based models and messy black-box machine-learning emulators, sharper in the telling than in reality. Two facts blur it.

First: “physics-based” climate models are not pure first principles either. They cannot resolve a cloud or a thunderstorm, because those are far smaller than the grid, so they fake the small stuff with parameterizations: simplified stand-ins whose knobs are tuned until the model reproduces the climate already observed. Climate scientists openly call this “the art and science of climate model tuning.” A model hand-tuned to match known climate, then trusted to predict an unknown one, is leaning on exactly the kind of fit that Oreskes warned cannot validate anything. The principled model has fudge in it too.

Second: when a simulation is itself too expensive to run thousands of times, scientists build a cheap statistical stand-in for the simulation, called an emulator or surrogate. Follow the chain: the world is approximated by a simulation, a model of the world, which is approximated by an emulator, a model of the model. Each rung is faster, cheaper, and a little more wrong. The AI weather models from the main page are simply the newest, most powerful rung of that ladder: surrogates that learned to mimic decades of atmosphere directly. “All models are wrong” turns out to be recursive. We routinely build wrong models of our wrong models, and the trick, every time, is knowing which lie is good enough for the job in hand.

Zoom all the way out and the day sits on a four-step staircase that the computer scientist Jim Gray called the four paradigms of science.

Coda

The aphorism, audited

Let us give the day’s slogan the same scrutiny we gave everything else. “All models are wrong, but some are useful” is true, but it has hardened into a thought-terminating cliché, a phrase people reach for to wave away the hard question instead of answering it. The real questions are the ones it skips: wrong how? Wrong for what? Some models are vastly wronger than others, and the difference between a useful map and a misleading one is the entire game. Statisticians like Andrew Gelman have pushed back precisely here. The aphorism, taken as a shrug, can excuse sloppy work and dull the urge to check.

George Box himself was no nihilist about it. The same papers that gave us “all models are wrong” warn just as hard against the opposite error: mistaking the model for the world, or “worrying about the wrong things” because you have fallen in love with your own elaboration. The discipline he was pointing at is not “models are wrong, so relax.” It is the harder thing this course is built around: calibrated judgment about the specific gap between this map and this purpose. A good modeler is not someone who has stopped lying. It is someone who knows exactly which lie they are telling, and exactly what it is good for.

The shift

From building a model to summoning one

The deepest post-2020 change is not any single result. It is a change in what a scientific model is made of. For four centuries, the usual recipe was explicit: choose idealizations, write equations, solve them. Since about 2022, a rival recipe has been spreading. Train a giant neural network on a staggering amount of domain data until it absorbs the statistical shape of the field, then fine-tune that one model for many concrete tasks.

These systems are foundation models, and importing them into science is one of the decade’s major stories. Wang and 41 co-authors’ 2023 Nature review, “Scientific discovery in the age of artificial intelligence,” reads like a flag planted on new territory. review

The flagship Earth-system instance arrived in May 2025: Aurora, a 1.3-billion-parameter model from Microsoft Research and collaborators, pretrained on more than a million hours of geophysical data. It is not tuned for one job; it is a general pretrained atmosphere that can be fine-tuned for air quality, ocean waves, tropical-cyclone tracks, and high-resolution weather. The paper reports that Aurora matches or beats numerical air-pollution simulations on 74% of targets, wave models on 86% of variables, and official five-day tropical-cyclone track forecasts across several agencies, including the U.S. National Hurricane Center basins. Its authors frame this as the first time a machine-learning model surpassed full operational tropical-cyclone forecasts up to five days. established

Notice what this does to the main page’s realism debate. Aurora is constructive empiricism industrialized: a model with no human-readable meteorology inside can still be empirically adequate for important forecasting tasks. The realist’s discomfort — a model that predicts without representing in a surveyable way — is now a deployed research program, not a classroom puzzle.

The same pattern is spreading. NASA and IBM’s Prithvi WxC, released in 2024 as open weights plus preprint, is a 2.3-billion-parameter weather-climate foundation model trained on MERRA-2 variables and tested across forecasting, downscaling, gravity-wave flux parameterization, and extremes. The gap between released and validated is the reason the hype filter exists. preprint

The map of the frontier

Thirteen dragons, sorted

The useful question is not “is this impressive?” The useful question has two axes: how established is it, and how much would it matter if it held up? The upper-right is the real revolution: big and solid. The upper-left is where the dragons live: enormous claims resting on thin ice.

Dispatch 1 · surrogates at the controls

The machine now runs the machine

Some frontier models are already load-bearing in the most literal sense: trusted with expensive hardware and unstable plasma. In 2022, DeepMind and EPFL published a reinforcement-learning controller that shaped the magnetic field of a real tokamak fusion reactor, holding plasma in target configurations on the TCV machine. In 2024, a follow-up used deep reinforcement learning to reduce the risk of tearing instabilities on the DIII-D tokamak. When a learned model is allowed to steer fusion plasma in real time, “just a model” is no longer a useful dismissal. hardware

The same dispatch also carries a warning. Physics-informed neural networks were pitched as a bridge between explicit equations and learned models: bake the differential equation into the loss function, then let the network learn a solution. The failure-mode literature shows the catch. A PINN can drive its residual loss down while the actual solution remains badly wrong. The model can satisfy every constraint you wrote down and still miss the world, because you did not write down all the constraints that matter. failure modes fixes early

Dispatch 2 · the realist counterattack

Discover the law — do not just fit it

If the black-box emulator is instrumentalism’s triumph, a quieter counter-movement fights for the realist. Its demand is simple: do not settle for a model that predicts; make the machine hand you a law you can read. This is symbolic regression and theory-guided discovery.

Three specimens matter. AI-Descartes combines symbolic regression with logical reasoning over background axioms and rederived laws such as Kepler’s third law from data plus theory. AI-Hilbert pushes the idea through polynomial optimization and formal certificates. FunSearch pairs a language model with an automated evaluator and evolutionary search; in Nature, it found new constructions for the cap-set problem and new bin-packing heuristics. That last case matters philosophically because the output is not a weight vector. It is an object humans can inspect, test, and prove against. FunSearch

The caveat keeps this from becoming a realist victory lap. Many systems still mostly rediscover laws we already had, which is powerful validation but thin discovery. The KAN line is similarly promising but unsettled: the 2024 Kolmogorov-Arnold Networks preprint proposed learnable functions on network edges as a more interpretable alternative to MLPs, and a 2026 Physical Review X follow-up argues for scientific-discovery uses. That is real momentum. It is still not proof that KANs generally beat well-tuned ordinary networks. promising KAN debated

Dispatch 3 · the result that should scare you

Leakage, and the reproducibility crisis in AI-for-science

The most important result in this appendix is a warning. In 2023, Sayash Kapoor and Arvind Narayanan published “Leakage and the reproducibility crisis in machine-learning-based science” in Patterns. Their survey found leakage errors across 17 fields, collectively affecting 294 papers. Data leakage makes a model look brilliant on paper because the evaluation data have partially entered the training pipeline.

This is Day 2’s replication crisis reborn for the AI age. When a model is evaluated on information it has effectively already seen, its “accuracy” is Borges’s one-to-one map of the training set: perfect and useless. established

Leakage inflates performance inside the data you have. Its twin problem appears when the world moves outside the climate the model saw. NeuralGCM’s authors report that the emulator stayed sensible at +1 K and +2 K sea-surface-temperature perturbations, but at +4 K its warming response diverged from expectations. ACE2, from the Allen Institute team, is explicit that sensitivity to separately changing sea surface temperature and carbon dioxide is not entirely realistic. This is the empirical heart of the realism worry: a pattern learner can be a superb interpolator and an unreliable prophet. extrapolation

Dispatch 4 · the new philosophy

What can an unreadable model let you understand?

The philosophers have not been idle. Emily Sullivan’s 2022 “Understanding from Machine Learning Models” gives the cleanest reframing: what blocks understanding is not black-box complexity by itself, but link uncertainty — weak evidence that connects the model to the target it claims to be about. A transparent model of the wrong target gives you nothing; an opaque model with strong empirical links may still support understanding. Opacity is not the core problem. Evidence is.

Others divide the terrain differently. Florian Boge separates opacity of training from opacity of representation and warns that deep learning can open a gap between discovery and explanation. Ramon Alvarado argues that AI is an epistemic technology, a tool that operates on knowledge itself and deserves its own epistemology. Sabina Leonelli keeps attention on the data ecosystem: these models inherit the opacity, bias, gaps, and politics of the datasets that feed them. The upshot is that AI-for-science may be forcing a third epistemic category: not theory, not experiment, and not quite ordinary simulation either. unsettled

Dispatch 5 · the audacious dragons

A virtual cell, a planet twin, and an AI that does science

Now the upper-left of the map: claims so large that, if they land, they redraw everything, and which today rest on thinner ice.

The AI Virtual Cell. In 2024, a large team spanning Stanford, the Chan Zuckerberg Initiative, Genentech, Google, and others published a Cell roadmap for building multi-scale AI models of living cells. The ambition is serious: a virtual cell that can support interpretable in silico experiments across biological scales. But the paper is a priorities-and-opportunities roadmap, not a built cell twin. roadmap

The planet twin. NVIDIA’s cBottle is a generative model for kilometer-scale global-atmosphere fields: a coarse global generator plus local super-resolution, trained over simulation and reanalysis data. The preprint is credible and useful. The “digital twin of the planet” aura around Earth-2 is marketing language doing work that the evidence has not earned. preprint twin hype

The autonomous scientist. Sakana AI’s 2024 “AI Scientist” preprint pitched an end-to-end automated researcher: idea, experiment, paper, and review. Independent evaluation in 2025 found a harsher reality: five of twelve proposed experiments failed because of coding errors, novelty judgments were poor, and some results were hallucinated or misleading. The result is not useless; the autonomous-scientist framing is exactly what the contested tag is for. deflated

Open questions

The experiments that would settle it

The right posture is not belief or dismissal. It is knowing which result would change your mind.

• Validated warming extrapolation. If an ML climate emulator reproduces a genuine warming response against held-out high-emission simulation or paleoclimate, the instrumentalist case strengthens. Right now, +4 K drift says caution.
• Leakage-audited replication. Independent, leakage-checked reproduction of a flagship foundation-model science claim would tell us how much of Kapoor and Narayanan’s warning applies at the top of the field.
• A genuinely novel discovered law. A symbolic-regression system that finds a new, subsequently confirmed law of nature would be a landscape shift. FunSearch’s cap-set result is the closest analogue in pure math.
• A validated clinical digital twin. A peer-reviewed, independently validated patient or organ twin that clears the National Academies bar would move medicine’s dragons out of the upper-left.