Superintelligence Governance Proposals Compared

Why Superintelligence Governance Is a Distinct Problem

Most AI governance discussion in 2026 is about systems that already exist: chatbots, hiring tools, content generators, autonomous agents operating within defined bounds. Superintelligence governance is a different kind of problem, because it tries to design institutions before the thing being governed exists in a stable, well-understood form. That makes it unusually speculative territory, and it explains why the proposals on the table look so different from one another: some assume a system resembling a slightly more capable version of today’s frontier models, while others assume a genuine intelligence explosion that could compress years of capability gains into months.

The Council on Foreign Relations has argued that 2026 could be a decisive year for how this plays out, pointing to converging pressures: frontier labs racing to scale training runs, governments drafting legislation in real time, and a growing public debate, catalyzed by the Future of Life Institute’s late-2025 statement, about whether the technology should be developed at the current pace at all. Understanding the landscape means being precise about who is proposing what, since the differences between an international treaty body, a domestic regulatory regime, and a voluntary corporate commitment are not just matters of degree, they are fundamentally different kinds of governance with different enforcement realities.

The International Oversight Model: An “IAEA for AI”

The most widely cited reference point in this space traces back to a May 2023 essay from OpenAI’s leadership, titled “Governance of Superintelligence” and credited to Sam Altman, Greg Brockman, and Ilya Sutskever. The proposal argues that any effort above a certain capability or compute threshold should be subject to an international authority empowered to inspect systems, mandate audits, and test for dangerous capabilities, modeled loosely on the International Atomic Energy Agency’s role in nuclear oversight. The appeal of the analogy is that it borrows decades of precedent: the world already has experience building an international body that inspects a dangerous, dual-use technology without owning it outright.

The gap between the analogy and reality is significant, however. The IAEA has legal authority under a multilateral treaty, the Nuclear Non-Proliferation Treaty, that took years to negotiate and depends on member states agreeing to intrusive inspections of their own facilities. No equivalent treaty for frontier AI compute exists yet, and the technical basis for inspection is arguably harder: nuclear material is physically trackable in a way that software and model weights are not, at least without additional hardware-level verification infrastructure that does not yet exist at scale.

Multinational Joint Ventures and Layered Institutional Proposals

A related but distinct family of proposals argues that oversight alone is not enough, and that the underlying development effort itself needs to be internationalized. Forethought has proposed a model it calls “Intelsat for AGI,” drawing on the precedent of the international satellite consortium Intelsat, in which multiple governments jointly develop and control a strategically significant technology rather than leaving it to a single national champion or a single company. Under this model, a multinational project would hold something like a short-term monopoly on frontier AGI development, using that position to slow a potential intelligence explosion, accelerate lower-risk applications, and withhold the most dangerous ones.

An academic paper circulated in 2025, cataloged on arXiv as preprint 2507.06379, takes a more layered approach, proposing four complementary institutions operating simultaneously: domestic frontier AI regulation within individual countries, an IAEA-style international inspection body, an NPT-style non-proliferation treaty specifically for frontier AI, and a US-led allied public-private partnership that pools resources and safety research among trusted actors. The paper’s core argument is that no single institution can address every risk, so governance needs to be built as a stack of complementary institutions rather than one grand design.

The Case for a Prohibition

Not every proposal assumes superintelligence should be built at all, at least not yet. On October 22, 2025, the Future of Life Institute released a “Statement on Superintelligence” calling for a prohibition on developing superintelligence until there is broad scientific consensus that it can be done safely and controllably, and strong public buy-in reflecting genuine democratic consent. The statement drew an unusually broad coalition of signatories, including AI pioneers Geoffrey Hinton and Yoshua Bengio, alongside technology figures, policymakers, faith leaders, and artists, reflecting a deliberate strategy of building a coalition that extends well beyond the AI research community itself.

A companion academic paper, posted to arXiv as preprint 2511.10783 under the title “An International Agreement to Prevent the Premature Creation of Artificial Superintelligence,” works through what a legally binding version of this idea might actually look like: a multilateral agreement establishing hard limits on training runs above a certain capability level, paired with a verification regime and a mechanism for lifting the prohibition once safety conditions are genuinely met. Compared with the oversight-body model, the prohibition model does not try to manage development, it tries to stop it, at least temporarily, which is a meaningfully different institutional design problem with a much higher political bar for adoption.

Compute Governance and Entity-Based Regulation

A more incremental family of proposals focuses less on new international bodies and more on the physical and organizational levers that already exist. Compute governance, a research area closely associated with GovAI, argues that computing power is currently the most trackable input into frontier AI development, since large training runs require detectable, expensive clusters of specialized hardware. Under this approach, governments require registration or licensing for training runs above a defined FLOP threshold, an idea already partially reflected in the EU AI Act’s obligations for general-purpose models trained above 10^25 floating-point operations. The approach’s own researchers have flagged a serious complication, though: distributed training techniques could let developers assemble frontier-scale compute across many smaller, less detectable clusters, which would undermine the assumption that large training runs are inherently visible.

Entity-based regulation, described by Dean W. Ball and Ketan Ramakrishnan of the Carnegie Endowment for International Peace, takes a different simplifying approach: rather than writing rules for every possible AI use case, regulate the small number of organizations capable of frontier-scale development directly, since at any given moment only a handful of labs plausibly possess the resources to train the most capable systems. Proponents argue this concentrates regulatory attention where it matters most and avoids the trap of writing rules so broad they burden ordinary software development. Critics counter that an entity-based approach can become quickly outdated as the list of capable developers changes, and that it does little to address risks from widely available open-weight models once a frontier capability diffuses.

What Frontier Labs Have Actually Committed To

While the international proposals above remain largely conceptual, the major frontier labs have already published their own voluntary safety frameworks, and these are the closest thing to operating governance that exists today, even though they carry no external enforcement. Anthropic’s Responsible Scaling Policy uses a tiered AI Safety Level system, ASL-1 through ASL-4 and beyond, modeled loosely on biosafety containment levels, gating deployment and even continued scaling behind capability evaluations. OpenAI’s Preparedness Framework tracks categories of severe harm, such as cyberattack automation or biological weapons assistance, and commits to specific safeguards once a model crosses defined risk thresholds. Google DeepMind’s Frontier Safety Framework follows a broadly similar structure, focused on testing for dangerous capabilities before and during deployment.

All three frameworks share a family resemblance: they emphasize misuse over misalignment, track broadly similar categories such as cyber, biological, and self-improvement risks, and use capability thresholds as the trigger for additional safeguards. What they do not share with any of the international proposals above is external verification. A lab can revise its own framework, as several already have, without any outside body confirming the revision was justified, which is precisely the gap that proposals like the “IAEA for AI” model are trying to fill.

Where US Federal Legislation Stands

Domestically, the clearest 2026 legislative signal is the Great American AI Act, a discussion draft released on June 4, 2026, by Representatives Jay Obernolte and Lori Trahan, described in policy press coverage as the most comprehensive bipartisan AI regulatory framework proposed in Congress to date. Unlike the international proposals discussed above, this is ordinary domestic legislation working through the standard committee process, and as of mid-2026 it remains a discussion draft rather than enacted law, meaning its provisions could still change substantially before any floor vote. Separately, the UK’s House of Lords held a debate in early 2026 specifically on a proposed superintelligence moratorium, an early sign that the prohibition-style argument advanced by the Future of Life Institute is gaining a hearing inside at least one national legislature, even without committing to specific binding text.

Frequently Overlooked Considerations

• Verification is the hard partNearly every proposal that involves inspection or licensing assumes training runs are detectable; distributed training research suggests that assumption may not hold as compute becomes more decentralized.
• Voluntary is not bindingLab safety frameworks like the Responsible Scaling Policy and Preparedness Framework can be revised unilaterally by the companies that wrote them, with no external body able to block a relaxation.
• Treaty analogies have limitsThe IAEA comparison borrows legitimacy from nuclear governance but skips over the decades of negotiation and the physical trackability of nuclear material that made that regime workable.
• Entity lists go staleRegulating today’s handful of frontier labs directly does not automatically capture a new entrant that reaches frontier-scale capability later, requiring the list of regulated entities to be actively maintained.
• Open-weight models sit outside most proposalsCompute governance and entity-based regulation both struggle to address risk once a frontier-level capability has already been released as open weights and can no longer be centrally controlled.
• Coalition breadth matters for prohibition proposalsThe Statement on Superintelligence deliberately recruited signatories well outside AI research, on the theory that a prohibition needs broad public and political legitimacy, not just expert consensus, to have any chance of adoption.
• Discussion drafts are not lawThe Great American AI Act remains a discussion draft as of mid-2026; treating draft legislative text as settled policy is a common and consequential misreading of where the process actually stands.

Glossary

Compute governance

Regulatory approaches that track, license, or restrict access to the computing power needed for large AI training runs, treating compute as the most detectable input into frontier development.

Entity-based regulation

An approach that regulates the small number of organizations capable of frontier-scale AI development directly, rather than writing rules for every possible AI application.

Responsible Scaling Policy

Anthropic’s voluntary framework gating model deployment and further scaling behind a tiered system of AI Safety Levels tied to demonstrated capability risks.

Frontier Safety Framework

Google DeepMind’s voluntary framework for evaluating dangerous capabilities in frontier models before and during deployment.

Superintelligence

A hypothetical AI system substantially exceeding human capability across most cognitively demanding domains, distinct from current frontier models.

Intelligence explosion

A scenario in which AI systems capable of improving their own successors trigger a rapid, compounding increase in capability over a short period.

FAQs

What is the “IAEA for AI” proposal?

It is a proposal, first articulated by OpenAI’s Sam Altman, Greg Brockman, and Ilya Sutskever in a May 2023 essay, for an international authority modeled on the International Atomic Energy Agency that could inspect and audit AI training efforts above a certain capability or compute threshold. No such treaty body currently exists, and building one would require multilateral negotiation similar to the Nuclear Non-Proliferation Treaty.

What is “Intelsat for AGI”?

Intelsat for AGI is a governance model proposed by Forethought, drawing on the precedent of the international satellite consortium Intelsat, in which multiple governments would jointly develop and control frontier AGI rather than leaving it to a single company or nation, aiming to slow risky development while accelerating safer applications.

What does the Statement on Superintelligence actually call for?

Released by the Future of Life Institute on October 22, 2025, and signed by thousands including Geoffrey Hinton and Yoshua Bengio, it calls for a prohibition on developing superintelligence until there is broad scientific consensus that it can be done safely and controllably, plus strong public buy-in reflecting genuine democratic consent.

How is compute governance different from entity-based regulation?

Compute governance focuses on tracking or licensing the computing power used in large training runs, treating detectable hardware clusters as the regulatory lever. Entity-based regulation, described by Carnegie Endowment researchers Dean W. Ball and Ketan Ramakrishnan, instead regulates the small number of organizations capable of frontier development directly, regardless of the specific compute they use.

Are Anthropic’s, OpenAI’s, and Google DeepMind’s safety frameworks legally binding?

No. The Responsible Scaling Policy, Preparedness Framework, and Frontier Safety Framework are voluntary, self-imposed, and self-enforced commitments. Each company can revise its own framework unilaterally, and no external regulator currently verifies compliance with these specific documents.

Is the Great American AI Act already law?

No. It is a discussion draft released on June 4, 2026, by Representatives Jay Obernolte and Lori Trahan, described as the most comprehensive bipartisan AI regulatory framework proposed in Congress to date, but as of mid-2026 it has not been enacted and its provisions could still change.

Why do compute governance proposals face a verification problem?

Compute governance assumes large training runs require detectable clusters of specialized hardware, but research on distributed training suggests developers could eventually assemble frontier-scale compute across many smaller, harder-to-detect clusters, which would undermine the core assumption the approach relies on.

Do any of these proposals directly address open-weight models?

Not effectively. Both compute governance and entity-based regulation are built around controlling access to frontier development before release, but neither offers a strong mechanism for managing risk once a frontier-level model has already been released with open weights and can no longer be centrally restricted.

References

• OpenAI, “Governance of Superintelligence” (May 2023)
• Forethought, “Intelsat for AGI: A Blueprint for International AI Governance”
• arXiv preprint 2507.06379, “Domestic Frontier AI Regulation, an IAEA for AI, an NPT for AI, and a US-Led Allied Public-Private Partnership for AI”
• Future of Life Institute, “Statement on Superintelligence” (October 2025)
• arXiv preprint 2511.10783, “An International Agreement to Prevent the Premature Creation of Artificial Superintelligence”
• GovAI, “Computing Power and the Governance of AI”
• Carnegie Endowment for International Peace, Dean W. Ball and Ketan Ramakrishnan, “Entity-Based Regulation in Frontier AI Governance”
• Council on Foreign Relations, “How 2026 Could Decide the Future of Artificial Intelligence”
• TechPolicy.Press, “Unpacking the Great American Artificial Intelligence Act of 2026”

For related coverage on this site, see our explainer on compute governance for a deeper look at FLOP-threshold regulation, our guide to frontier model safety evaluations for how labs actually test dangerous capabilities, and our analysis of open-weight model governance for why released models resist the controls discussed above. Readers interested in the global regulatory picture may also want our comparison of global AI ethics standards and our piece on AI personhood and legal standing. For a narrower, more technical look at how compliance obligations get enforced day to day, see our companion piece on automated compliance and policy as code.