Open letter from DeepMind CEO Demis Hassabis arguing that AGI is probably only years away, the commercial race has outpaced our understanding of frontier capabilities, and the US should establish a FINRA-style self-regulatory body to evaluate and gate deployment of frontier models.
AGI as the Next Epochal Technology
Hassabis frames AGI as fundamentally different from the internet or mobile - it is comparable to the discovery of electricity or fire. Impact could be “10x of the Industrial Revolution at 10x the speed,” enabling drug discovery, clean energy advances, novel materials, and potentially a post-scarcity future. The core premise: we have found a way to make sand think, and the consequences will be unlike anything humanity has managed before.
He acknowledges this is a view he has held his whole career, and believes the technology could be “one of the most beneficial and transformative technologies ever invented” if built and deployed responsibly. The positive vision is not hedged - it is stated with conviction.
The Geopolitical Race Outpacing Safety
The sobering counterpoint: we are locked in an “extremely intense, multilayered commercial and geopolitical race” where advances on the frontier are outpacing our understanding of the technology. Nobody knows what happens next, and even experts disagree.
Specific risks identified:
- Cybersecurity - frontier models already pose challenges here
- Biological threats - expected to emerge as capabilities advance
- Nuclear risks - also on the horizon
- Agentic and self-improving systems - robust safeguards will be needed to maintain control of increasingly autonomous, recursively self-improving models
Hassabis argues the field and wider society are not currently giving themselves the time and space to get this right. Proceeding with “cautious optimism” is the sensible strategy - policy that promotes innovation while incentivising responsibility and security, international collaboration on key safety issues, and careful deployment for societal benefit.
The FINRA-Inspired Standards Body
The concrete proposal: a new US Standards Body modelled on FINRA (Financial Industry Regulatory Authority) - a federally overseen public-private partnership or self-regulatory organisation. A board of independent technical experts and open-source representatives would govern it. Funding would be substantial and mostly from industry to attract world-class talent and provide compute for large-scale testing.
The Standards Body would work with federal agencies and US National Labs to conduct testing in national security areas. It would define what qualifies as a “Frontier-class” model via periodically updated benchmark thresholds. Organisations with Frontier-class models become “Frontier Labs” and are encouraged to adopt best practices: model cards, strong cybersecurity, personnel vetting, and adequate safety/security research resourcing.
Assessment Protocol and Enforcement
The process is designed to be phased and adaptive:
- Voluntary phase - Frontier Labs share models up to 30 days before release for review
- Mandatory phase - once the protocol is proven effective, Frontier Models must pass assessment to be deployed in the US market
- Post-release - Labs work with the body to address critical vulnerabilities
Assessments would cover cybersecurity, biological threats, and other high-risk domains. Specific agentic AI tests would check for attempts to bypass safety guardrails, signs of deception, watermarking of AI-generated images, and human-readable output tokens for reasoning transparency. Evaluations would be updated quarterly, with saturated benchmarks deprecated. The body should eventually build independent held-out tests to prevent overfitting, and promote a third-party auditor ecosystem.
Key design choice: non-frontier models are exempt - startups and academia face no burden. The framework applies regardless of country of origin or open/closed status. Being a Frontier Lab carries prestige and is open to any organisation that meets the criteria. If the situation demands, the framework can be ratcheted up, including coordinating a slowdown among Frontier Labs.
Post-Scarcity and the Unwritten Future
The letter closes on a philosophical note. Even if the technical governance problems are solved, deeper questions remain: what economic models help everyone thrive in a post-scarcity world? What values, meaning, and purpose look like? How the human condition itself might change? Hassabis explicitly says these cannot be left to technologists alone - they require every part of society.
The final framing: the future is not yet written, and this precious window before AGI arrives must be used to shape the technology for the benefit of all humanity.
Lessons worth keeping
- Hassabis publicly commits AGI being “a few short years away” - this is the strongest timeline signal from a DeepMind leader, and it has regulatory implications baked in
- FINRA as a governance model means: industry-funded, expert-led, with explicit power to gate market access - not an academic advisory board
- The 30-day pre-release review window is the proposed concrete timeline for mandatory evaluations
- The framework explicitly exempts non-frontier models (startups, academia) to avoid over-regulation of smaller players
- Agentic AI tests for deception, guardrail bypass, and reasoning transparency get explicit mention as priority evaluation areas
- The letter suggests the Standards Body could eventually coordinate a “slowdown” if risks escalate - this is the most specific enforcement mechanism proposed
- Post-scarcity economics and meaning are acknowledged as open questions beyond AI’s technical domain - Hassabis defers to broader society on these
Related
No directly overlapping notes in the wiki yet. This is the first entry dealing specifically with AI governance / regulation policy. Related tangentially: How to Automate AI Evals (Correctly) covers the technical eval side.