“We must slow the pace at which we improve the capabilities of AI models.” That sentence came from Anthropic CEO Dario Amodei this weekend. And he may be right. But I keep coming back to a more uncomfortable question: can we actually slow AI down?
Amodei proposes slower capability growth, permanent access for independent evaluators and wider industry and global coordination.
There is a serious safety argument. More capable autonomous systems create risks we do not fully understand. Amodei’s warning should not be dismissed as theatre simply because Anthropic has commercial interests.
But safety does not exist outside economics.
Safety buys time.
So does slowing down.
Frontier AI is a strange business: extraordinary revenue growth coupled with extraordinary capital requirements. OpenAI reported more than $20 billion in annualised revenue for 2025, while describing compute as the scarcest resource in AI. xAI’s public filing, after its combination with SpaceX, recorded a $6.36 billion operating loss for its AI segment in 2025.
Anthropic is the important counterexample to any simplistic claim that “AI companies cannot make money”. Its growth has been exceptional, and reporting in 2026 indicated it was approaching its first quarterly operating profit.
Time for revenues to catch up with infrastructure commitments. Time for inference costs to fall. Time for business models, regulation and liability to become clearer. Potentially, time to reach public markets with better numbers.
This does not invalidate the safety case. It means safety, corporate strategy and economic pressure can point in the same direction.
But what does “slow down” mean once the weights are out?
Frontier laboratories can be regulated. Large training runs can be monitored. Compute clusters, chip supply and commercial APIs have identifiable operators. Open weights are different. Once model weights have been distributed globally, there is no reliable central mechanism that recalls every copy.
The model can be copied, modified, specialised, quantised and run locally. Commercial providers may pause; the diffusion of existing capability does not.
Imagine a freely available model with the cybersecurity ability of the world’s best human teams. Four major labs could stop training tomorrow. The capability would still exist.
A malicious actor might no longer need to recruit a rare team of exceptional experts if parts of that expertise can be instantiated repeatedly in software. Defenders receive the same technology, of course: AI can identify vulnerabilities, monitor systems, develop patches and accelerate incident response. Research does not yet justify certainty about which side gains the lasting advantage.
Intelligence is not access
Even a very capable AI does not magically “hack everything”. A digital system is not vulnerable merely because it is built from ones and zeros. To affect a power station, aircraft, factory or hospital, an attacker still needs a causal path: connectivity, credentials, vulnerable components, permissions, compromised hardware or people.
If we may eventually lose control over who has intelligence, we must retain control over what intelligence can reach.
This shifts the centre of AI safety. Model governance remains necessary, but it is not sufficient. Critical systems must use least privilege, isolation, authentication, monitoring, human authorisation and graceful failure. The security perimeter moves from the model to the world around it.
The second inversion:
intelligence may become abundant
Open and open-weight models are improving. Distillation, quantisation and specialised architectures already move useful capabilities from data centres onto local devices. Follow that trajectory far enough and today’s frontier systems may eventually look astonishingly inefficient.
One day, a car key may hold more useful specialised intelligence than today’s largest models—not a giant general model, but compact intelligence shaped for a narrow job.
If intelligence becomes abundant, scarcity moves down the stack.
The International Energy Agency expects global data-centre electricity consumption to rise from around 485 TWh in 2025 to roughly 945 TWh by 2030. Electricity use in AI-focused data centres is projected to more than triple over that period. That is not a distant philosophical limit. It is infrastructure.
Energy is not a practical master switch for every model: smaller local systems may need very little of it. Compute controls are most effective where training is concentrated and visible, and less effective when inference becomes cheap and distributed.
So the ultimate AI safety problem may not be how to stop intelligence. It may be how to build a world that remains safe when intelligence can no longer be stopped.
And perhaps the ultimate constraint on AI will not be regulation.
It will be physics.