The Machine Stops: Science and Its Limits

Units of its currency — ideas — are disseminated via lecture and shared via chat; like cryptocurrencies, their value is relational rather than a proxy for some relationship to reality. The answer, to paraphrase the Mandalorian, is that this is the way. Could the same be true for us? Both were onto something, Strevens argues, especially in their recognition of the degree to which scientific thought is alienating, sometimes brutally so: it is just plain hard to be so skeptical, or to adhere to Kuhn’s “normal science.” But what each failed to grasp, according to Strevens, is why anyone would spend their lives in a laboratory, desperately pursuing these alien lines of thought. Above her, beneath her, and around her, the Machine hummed eternally.” To Vashti, the Machine offers serenity and certainty in equal measure. Along the way, Strevens provides a pocket history of the field of science studies, starting with a comparison of two of its figureheads: Karl Popper and Thomas Kuhn. While the roots of “knowledge production” run deep, it gained popularity through the work of the economist Fritz Machlup. The machine and anything we put into it must be “sterilized,” as Strevens puts it. Readers were returning to “The Machine Stops” long before COVID-19 and Zoom turned so many of us into supine hermits binging media pre-selected by artificial intelligence. In an age of crises — epidemiological and climatic, political and cultural — the crisis of expertise may be the most urgent, not least because its resolution will affect the others. Like Popper and Kuhn, Strevens is a “methodist,” a believer in rules. It is all around us yet difficult to see, let alone explain. Noting the existential threat posed by climate change, he limits himself to the claim that the knowledge machine is “our best chance of salvation” before offering tips for “ensuring that it purrs — efficient, responsive, dynamic, and strong.” Once again, the book is more a user manual than a sermon. (The exceptions, like a lengthy retelling of Romeo and Juliet as a debate over the caloric theory of heat, can presumably be chalked up to his being a philosopher.) So we get the testing of general relativity by an avowed partisan, Arthur Eddington, as a case of the iron rule in action. Some of science’s problems (with reproducibility, say, or with biases that affect who is or isn’t seen as a researcher) are with the application of the iron rule — or with the iron rule itself. The knowledge machine offers us one path forward, explaining how science works and how to care for it; but to explore other paths, new metaphors might be needed. The book mirrors its subject: rather than intervene, or celebrate, or worry, it offers explanatory data. In this imagined world of isolated togetherness, heaven is a place. For reasons explored by Strevens’s colleague, the historian Matthew Stanley, Eddington really wanted Einstein to be right but never said so, at least not in print. Either way — whether we’ve got Kuno Problems or Vashti Problems, whether the knowledge machine needs a tune-up or the metaphor needs the scrap heap — there’s enough at stake to reconsider, if not reimagine, how we talk about science. But what if it isn’t? Yes. “Cannot you see,” he asks, “that it is we that are dying, and that down here the only thing that really lives is the Machine? Thus, even at the end of the book, in a final chapter on “Care and Maintenance of the Knowledge Machine,” Strevens seems loath to cross over from is to ought. If recent debates are any indication, this is the world in which we find ourselves, and our metaphors of solidity aren’t capturing what is ailing scientific experts. Popper identified their skepticism, and Kuhn their allegiance to a paradigm. And what a brilliant description it is! Strevens (mostly) does the same, offering up historical examples of scientists checking their biases at the proverbial door. Forster offers a clue. Unlike them, he thinks science’s rules limit writing, not thinking. From reliance on smartphones to hope for a vaccine, we assume science works; when it doesn’t, we blame the operators, not the machine. Where is our Machine hiding? Some say the internet, others say social media or digital surveillance. Call them Vashti Problems, since even true believers can’t ignore them for long. It is the loss of meaning, even of truth, as the ideas stop circulating. The second kind of threat is entirely different. As long as scientists keep making observations that can — at least in principle — be contested, then they’ll keep producing interesting ideas and technological marvels. Eventually, Kuno delivers a cryptic prophecy: “The Machine stops.” At first, Vashti finds this unintelligible. Endogenous to the machine, these threats are signs of systemic failure. Cracks appear, first as heresies whispered by Vashti’s son Kuno — in person, since they cannot be shared through the Machine. The problem with Popper and Kuhn, he says, was that they were looking in the wrong place: the minds of scientists. Doubling down on knowledge, not to mention truth, is attractive precisely because knowledge feels solid, even tangible — the kind of thing you could produce with a machine. But this material metaphor, embedded in phrases like “knowledge production” and “social construction,” should give us pause. He is the author of The Scientific Method: An Evolution of Thinking from Darwin to Dewey (Harvard University Press, 2020). This isn’t a defense of irrationality — nor is it a condemnation. The Knowledge Machine thus addresses two foundational questions in the history and philosophy of science — “Why is science so powerful?” and “Why did it take so long to arrive?” — with one answer: the iron rule. But these analogies ultimately fail: each of today’s technologies is criticized, whereas Forster’s Machine never was — until Kuno. Our equivalent must be beyond reproach, at least in polite company; it must shape daily life via hidden processes shrouded in myth. In The Production and Distribution of Knowledge in Society (published in 1962, the same year as Kuhn’s work on paradigms), Machlup cast knowledge in terms of production and consumption, circulation and extraction. As in the parable of the blind men and the elephant, Popper and Kuhn, he argues, have identified two aspects of modern science but have failed to grasp the machine itself. M. But strangely, he does. Answering these questions requires a hard look at the machine itself, either to diagnose and address the issue or to ask whether the knowledge machine ever worked how we thought it did. The rule sidelined centuries of debate about higher, often religious questions in scientific communities by narrowing the field of acceptable evidence. ¤
But sometimes, description is not enough. Forster from over a century ago, about a future that feels uncannily like our present. We could call these Kuno Problems, since — at least early on — critiques of the Machine were not about its failure to provide ideas or food but about the pervasive, negative effects of such provision. The dangers are real, and not limited to climate change. Solutions to Kuno Problems — to the manufacture of ignorance — aren’t going to come from tinkering with the knowledge machine, since these problems are less about how it works and more about its effects, or its limits. So when the Machine stops, proving Kuno right, it is not just the loss of material comfort that horrifies Vashti and the others. Until it doesn’t. Rather, it is despite what goes on in scientists’ heads — despite their hopes and dreams, biases and guesses — that the knowledge machine hums along. The knowledge machine keeps humming. Called “The Machine Stops,” the story conjures a world of individual humans isolated in such rooms who pass the time by streaming lectures and videoconferencing as the titular Machine tends to their every need. Enter Strevens and his machine. It’s a compelling idea, especially in the face of organized denial and post-truth cynicism. The Machine, omnipresent and omniscient, is reality, if not God. Is science’s supposed insulation from society responsible for its triumphs and its travails, including not only organized denialism and declining public trust but also crises of replication and inclusivity? Because of the iron rule, which requires that “the grounds of many of the experimenter’s crucial assumptions, being partially or wholly subjective, are cut away.” Would papers be more complete if those assumptions and other human dimensions were included? For Strevens, the answer is no. Might the sterilization Strevens identifies be part of the problem, even as it accounts for so many solutions? We created the Machine, to do our will, but we cannot make it do our will now.” Gradually, the cracks widen: in place of music, jarring noises echo; at night, beds fail to appear. What’s the best analogue for the Machine in today’s world? It is a world in which everything — music, food, even your bed — is summoned by the click of a button. Forster’s eerie prescience isn’t even new. If this feels too close to home, you’re not alone. But we need sermons now. With the rule in place, the knowledge machine creaked into motion — and hasn’t stopped. ¤
For starters: Does science produce knowledge? This combination of power and opacity both inspired Vashti’s devotion and hid the spanner in the works. This puzzling austerity explains both Strevens’s metaphor (machines don’t care if we find them intuitive) as well as his subtitle: “How Irrationality Created Modern Science.” What makes the machine irrational is that it would make a lot of sense for scientists to share their hopes and biases (beyond a disclosure of conflicts of interest) — but doing so would gum up its gears. What happens when the foundation wobbles, when the machine makes jarring noises or starts getting second-guessed — or ignored? Strevens’s argument is breathtaking in its simplicity: control how people argue and you don’t just get better arguments, more agreement, and less bickering — you get the truth, or something close to it. If so, getting a grip on the machine should help us troubleshoot the jarring noises it is emitting. The first is a familiar exogenous threat: denial or distrust aimed at the knowledge machine even when it seems to be running perfectly. And, by applying only to public statements, it did not run afoul of the unavoidable biases of private thought. If science isn’t to be captive to the market, to the up-votes and down-votes of capitalism or social media, then we will need to rethink whether it is a machine for producing a tradable good. After all, the machine stops. ¤
Henry M. Certain moments call for prescription, ethics, normative arguments, or aesthetic distinctions — iron rule be damned. What if science actually suffers because, in publishing it, we omit what motivates us to pursue it? ¤
The Knowledge Machine is not just an argument about modern science. Like Machlup’s idea of “the knowledge economy,” this framing blurred the process of inquiry with its products, including the papers spit out by Strevens’s machine. Or rather, in the journal articles that flow from laboratory work. Such metaphors, figuring knowledge as a tradable good to be produced and circulated, bolster our sense that science is solid — a stable thing that buttresses reasoned debate. Each captures the uncomprehending dependence at the center of Forster’s story. It describes a rule that it also follows. The Knowledge Machine is neutral, a description of what’s in the box rather than a prescription for what to do with it. FEBRUARY 22, 2021
“IMAGINE, IF YOU CAN, a small room, hexagonal in shape, like the cell of a bee.” So begins a story by E. ¤
In the laboratory. Why? But how will we be able to tell when it stops? In place of work, the Machine offers material and intellectual sustenance. The knowledge machine itself is in trouble, facing at least two kinds of threat. Strevens is not a scientist, and thus does not have to follow the rule himself. Assuming that the misinformation peddled by merchants of doubt is just that — misinformation — then their assault on science has less to do with the machine than with the politics of expertise in an age of alternative facts. While Strevens is interested in what a scientist thinks, he doesn’t view their individual thoughts as explaining why science is so successful — or so recent. For the story’s main character, Vashti, it turns life into a reassuringly predictable cycle: “[S]he made the room dark and slept; she awoke and made the room light; she ate and exchanged ideas with her friends, and listened to music and attended lectures; she made the room dark and slept. Readers may wonder why useful information, such as spelling out a researcher’s desires or hidden bias, has no place in scientific publications. A scientist can think whatever she wants — so long as she adheres to the iron rule when publishing. Until it isn’t. But would they be scientific? Cowles is a historian of science and medicine based at the University of Michigan. These are the outputs of what the philosopher of science Michael Strevens, a professor at New York University, calls “the knowledge machine.” Strevens’s Machine, the engine of modern science, runs on a single principle, a kind of speech code that he dubs “the iron rule of explanation.” This rule doesn’t limit what or even how scientists think but how they argue: “[I]t directs scientists to resolve their differences of opinion by conducting empirical tests rather than by shouting or fighting or philosophizing or moralizing or marrying or calling on a higher power.” According to Strevens, the iron rule is the difference between science and non-science, and its introduction in the 17th century drove what is often called the Scientific Revolution. Like the Machine, modern science is both essential and obscure.