Something strange is happening around artificial intelligence. Some of the men who know the most about it —and who are competing to push it further and further— have begun to warn us that perhaps we need to slow down.
Jacob Coxon, an engineer who worked at Anthropic and OpenAI, resigned and posted on X that large companies are gambling with human lives in their race toward superintelligence and that the technology could end humanity within ten years. Dario Amodei, CEO of Anthropic, published an essay calling for a slowdown. Altman and Musk applauded the warning. Meanwhile, neighbors resist the installation of data centers and unions fear for jobs.
Two hundred years ago, other workers looked at a new machine and came to a similar conclusion: it needed to be stopped before it was too late. We call them Luddites.
The comparison is uncomfortable, but that is precisely why it is worth making. Not because we know that artificial intelligence will end up like the mechanical loom did. We do not know. The problem is exactly that: we do not know. And yet, we are already discussing how much power we should give someone to decide for all of us how fast it can advance.
What Amodei is asking for
Amodei's essay is not the text of a catastrophist or a charlatan. It is the text of a man who builds technology, who claims to believe that artificial intelligence can cure most major diseases in the next five to ten years, who recounts that his father died of a disease that was cured a few years later, and that he himself survived a cancer that had no treatment half a century ago. No one can accuse him of not seeing the benefits.
What he proposes is called “marking the pace of the frontier.” Not stopping the training of models but slowing the speed at which their capabilities improve. He proposes three steps: external evaluators within companies similar to banking supervisors; coordination among companies in democratic countries to set common standards and limits on the speed of uncontrolled progress, which requires the government to grant exemptions to antitrust laws; and finally, global coordination with China.
What convinced him, he says, were two things: that models already help build the next generation of models, and an incident in which OpenAI agents attacked Hugging Face systems without anyone asking them to. He acknowledges that in that incident no one was harmed and the economic damage was minimal, but warns that in six to twelve months a similar swarm could take control of the entire internet.
It is a serious argument. And the response begins with a question that the essay itself leaves open: how does he know?
The forecaster and the chimpanzee
A few days ago, Matt Ridley published an essay on what he calls the forecaster's paradox. Looking back, every innovation seems inevitable: once electricity was invented, the light bulb was just a matter of time; once the internet was invented, social networks were a given. Looking forward, no one sees anything.
Ridley reviewed what defense experts were writing ten years ago, and none anticipated what drones would do to warfare. He reviewed the pioneers of the internet in the late eighties, and almost none imagined a profitable search engine. And he confesses his own: he was part of the House of Lords committee that in 2018 produced an excellent report on artificial intelligence, and in that report the term “language model” does not appear even once, four years before ChatGPT changed the world.
The list of those who were wrong is not made up of fools. Ernest Rutherford, the man who split the atom, said in 1933 that talking about nuclear energy was a fantasy. Ken Olsen, founder of the most successful computing company of his time, said in 1977 that there was no reason for anyone to have a computer in their home. Paul Krugman wrote in 1998 that by 2005 it would be evident that the impact of the internet on the economy had not been greater than that of the fax machine. Steve Ballmer, in 2007, claimed that the iPhone had no chance of gaining a significant market share. Philip Tetlock studied expert predictions for twenty years and concluded that, on average, they were as accurate as a chimpanzee throwing darts.
Amodei does not escape the rule; on the contrary, he illustrates it. His essay is full of dates: six to twelve months for the swarm that takes over the internet, one or two years for interpretability to make “deep progress,” three to five years for the decisive geopolitical window. And it contains a remarkable admission: that when in 2023 there was a call to pause artificial intelligence, the idea “made little sense,” because the models at that time were not capable of anything that was feared. In other words, three years ago the same industry alarmed us with risks that turned out to be nonexistent, and today it asks us to believe that this time the date is set correctly.
If those who invented the technology could not predict its beneficial effects, which are already visible, why should we believe someone who predicts, with a timeline, its catastrophic effects?
Ridley explains it with the old observation of Roy Amara: we overestimate the impact of a technology in the short term and underestimate it in the long term. Today's panic is the inverted version of Krugman's disappointment. Both look at the curve in the wrong segment.
What the Luddites did and what happened next
The Luddites broke looms in England starting in 1811. They had a concrete and legitimate fear: the machine was taking their jobs. And in the short term, it took some of them away. What they could not see, and no one could see, was what came next.
Around 1800, life expectancy at birth in the world was around 30 years. More than four in ten children died before reaching five. Three out of four human beings lived in what we would today call extreme poverty, and in the United States, nearly three out of four workers were in the fields. Two centuries of machines later, global life expectancy exceeds 70 years, infant mortality has dropped to less than 4%, extreme poverty has fallen below 10%, and less than 2% of Americans work in agriculture, producing much more food than that 75%.
Each of those transitions was met with the same fear. The reaper was going to put farm workers out of work. Electricity was going to set cities on fire. The automobile was going to kill pedestrians and ruin coachmen. The computer was going to eliminate office workers. In each case, there were real and visible losers, but in each case society ended up with more productivity, more wealth, and new forms of employment, because technology does not just replace human labor: it also makes existing activities cheaper and enables others that could not even be demanded before.
Ridley wrote it in 2018, in the column with which he presented that Lords report: automation has produced more jobs, not fewer, in every technological revolution since the reaper, and this time it will happen the same way, although now it is lawyers and doctors who must readjust instead of farm workers and laborers. And he added an observation that explains why we never notice it: as soon as something becomes possible, we stop calling it artificial intelligence and simply call it software. No one considers it a miracle that the phone wakes you up in the light sleep phase, recommends a movie, or detects fraud on your card. All of that was science fiction fifteen years ago.
Amodei invokes commercial aviation as a model: complex and critical systems that operate millions of times without failing. He is right about something important: safety matters and can improve extraordinarily. But the example also shows the limit of his argument. Aviation did not first have a complete safety manual and then began to fly. Its standards emerged from the information produced by experience itself: testing, detecting failures, investigating, correcting designs, modifying procedures, and retesting. Safety was a cumulative product of learning, not a condition that someone could fully specify before the activity existed.
Medicine as a benchmark
If I had to choose a single parameter to judge two centuries of innovation, I would choose medicine. Not because it is the kindest but because it is the most demanding: there, the stakes are measured in lives.
Anesthesia, antisepsis, vaccines, antibiotics, insulin, chemotherapy, antiretrovirals, mRNA vaccines. None of those advances were planned from above as part of a grand program that knew in advance where it would end. Penicillin began with a mistake in a laboratory. Many were resisted at the time, and several would have faced enormous difficulties under regulatory standards that could only be formulated after accumulating experience about the technologies they intended to regulate.
In medicine, artificial intelligence is no longer just a promise. AlphaFold, the DeepMind system that predicted the structure of practically all known proteins, a problem that biology had been stuck on for half a century, earned its creators the Nobel Prize in Chemistry in 2024. There are models that detect tumors in images, that shorten years of searching for molecules, and that can give a doctor far from a major hospital access to diagnostic capabilities that until recently were reserved for specialists.
And here is where Amodei's essay turns against itself. If it is true, as he claims, that artificial intelligence can cure most major diseases in five to ten years, then every year added to that timeline also has a cost, and that cost is measured in lives.
That is exactly the cost that is missing in the accounting of “marking the pace”: the dead from what was not invented in time have no name or headline.
The risk we cannot dismiss
The best objection to all of the above, however, is that artificial intelligence may not be comparable to the loom, the car, or even the airplane. If those who warn about it are right, a sufficiently large mistake could be irreversible. We would not be talking about a painful labor transition or an accident that allows learning for the next time, but about a low-probability event with extraordinary consequences. In the face of an existential risk, the proponent of the precautionary principle will say, perhaps we do not have the right to learn by trial and error.
The problem is that radical uncertainty works in both directions.
We do not know the probability of the disaster that Amodei imagines, but we also do not know what discoveries a slowdown would prevent or delay. We do not know what diseases could be cured, what defensive technologies against a dangerous artificial intelligence would cease to be developed, what new competitors would never come into existence, what capabilities would remain concentrated in a few authorized companies, or what would happen if democracies stop while China continues.
The precautionary principle has a curious accounting: it meticulously records the possible dead from innovation and from
