AI Alignment Is a Governance Problem. The Architecture Is Being Built Right Now

In October 2025, OpenAI completed a restructuring two years in the making. Following the November 2023 crisis in which a nonprofit board fired and then had to reinstate its CEO under pressure from employees and investors, the organization redesigned its governance architecture. The nonprofit became a Foundation that holds sole power to appoint and remove all directors of the Public Benefit Corporation; PBC directors are barred from considering stockholder pecuniary interests on safety and security matters; and the Safety and Security Committee remains a committee of the nonprofit, with authority to impose mitigations up to halting model releases.¹

Those mechanisms exist on paper. The practical enforcement backstop is the attorneys general of California and Delaware, who extracted these commitments as a condition of approving the conversion.² The structure has not been tested against a real commercial pressure gradient. The question it raises is not whether the mechanisms were well designed, but whether inspection authority that sits largely inside the regulated entity, coupled with a consequence backstop confined to the attorneys general of California and Delaware, is sufficient when the organization and its investors both have strong incentives to override the safety judgment. That question is not a legal question. It is not an engineering question. It is a governance question, and every serious attempt to align AI behavior with human values eventually runs into the same wall.

The technical work on alignment matters. The serious labs invest heavily in it and the progress is real. The problem is that this work is downstream of questions engineering cannot settle: whose values get encoded, who holds authority to modify those choices, and who is accountable when things go wrong. Those questions are being answered right now, by default, in the absence of deliberately designed structures to answer them intentionally.

Why governance failures follow a structure

Every leading alignment technique presupposes a governing authority that has already answered three prior questions: whose values count, who sets the target, and who certifies the result.

Constitutional AI, the technique Anthropic uses to train its models against an explicit set of principles, draws those principles from the United Nations Universal Declaration of Human Rights, Apple’s terms of service, and DeepMind’s conduct rules: a synthesis chosen by the developer.³ When Anthropic ran a parallel experiment asking roughly 1,000 Americans to draft their own version through an online deliberation platform, the result had only about 50% conceptual overlap with the company-authored version. The public version leaned toward objectivity and accessibility rather than the developer’s preferred risk language.⁴ The same technical method, driven by a different governing body, produced a measurably different definition of aligned behavior. The governance choice was already embedded in the technical method before the engineering began.

Three structural failures recur across every domain where this pattern has played out.

The first is the authority gap. The UK AI Security Institute holds roughly £100 million in total public funding.⁵ Alphabet announced 2025 capital expenditure guidance of $91–93 billion, driven primarily by AI infrastructure investment.⁶ The public evaluation apparatus cannot independently probe safety claims made by organizations operating at that scale. Without inspection rights, any governance framework is only as reliable as each regulated party’s willingness to disclose accurately.

The second is regulatory capture by design. The Seoul Frontier AI Safety Commitments were brokered by the UK and Korean governments with company input.⁷ Responsible scaling policies and the capability thresholds that define when a model is “too dangerous to deploy” are set by the same organizations making deployment decisions. The Federal Aviation Administration’s decision to delegate Boeing’s 737 MAX safety evaluations to Boeing’s own engineers is now studied as a structural governance failure: the regulated party audited itself, evaluations lacked independence, and safety-critical systems were not flagged as such.⁸ The analogy to frontier AI self-evaluation is direct.

The third is the legitimacy deficit. Researcher Iason Gabriel’s careful formulation of the alignment challenge identifies the core problem precisely: the goal is “fair principles for alignment, that receive reflective endorsement despite widespread variation in people’s moral beliefs.”⁹ No internal corporate review process, however rigorous, produces that endorsement. When the entity setting alignment targets is also the entity certifying whether those targets were met, the resulting governance is advisory at best.

What durable governance requires

The Food and Drug Administration’s premarket approval framework provides the clearest institutional comparison.¹⁰ A drug cannot be marketed until cleared. The burden of proof falls on the applicant. The FDA has statutory authority to inspect manufacturing and access proprietary data. The framework took a catastrophe to build: the Federal Food, Drug, and Cosmetic Act was passed in 1938 after a sulfanilamide formulation killed approximately 107 people, and the resulting institution can actually stop a harmful product from reaching the market.

The Nuclear Non-Proliferation Treaty arrived more than two decades after the first nuclear weapon was deployed.¹¹ Its enforcement is not primarily the treaty text but the combination of International Atomic Energy Agency inspection authority, which treaty signatories grant by accession, and Security Council sanctions. Two decades of proposals and partial measures preceded the NPT. They did not produce durable governance. The NPT did, for compliant states, because it combined inspection with consequence.

The optimistic case for scalable technical oversight holds that if behavior can eventually be verified mechanically, the governance load shrinks. That may prove true at the verification layer. It does not settle who defines the target or who is accountable when the mechanism fails. Those remain governance questions even in the optimistic scenario.

Neither model maps cleanly onto AI, and transplanting either wholesale would raise barriers to entry high enough to further concentrate development among the largest labs. These tradeoffs are real. The answer is to build institutional capacity (trained evaluators, statutory authority, and clear standards) while the architecture is still young enough to be shaped.

The urgency of the next twelve months

The EU AI Act is the most ambitious AI governance framework enacted to date.¹² Its General-Purpose AI model obligations became applicable on August 2, 2025 and are being applied now. As originally enacted, high-risk system compliance requirements for standalone systems were scheduled for August 2, 2026, and for systems embedded in regulated products for August 2, 2027.

Those deadlines face deferral, pending formal adoption expected before the original August deadline. On May 7, 2026, the European Parliament and Council reached a provisional agreement under the Digital Omnibus package deferring standalone high-risk system compliance to December 2, 2027 and embedded-product systems to August 2, 2028.¹³ The proximate causes are a textbook governance failure: competent authorities had not been designated, harmonized standards the compliance framework depends on had not been published, and coordinated industry pressure found that the political will to enforce the original timeline was softer than the text suggested.

The deferral sharpens the argument rather than weakening it. The precedents taking shape in the ongoing standards process (what constitutes adequate evidence of compliance, how technical expertise from labs is weighted against independent assessment, whether inspection rights survive the negotiation) will define how binding AI governance operates for a generation. The deferral itself is evidence of what happens when that infrastructure is not built before the political window closes.

That window is not fully closed. The EU AI Office’s enforcement powers over providers of general-purpose AI models come into force on August 2, 2026: the authority to request documentation, conduct model evaluations, and impose fines of up to 3 percent of global annual turnover.¹⁴ This is a live test of whether binding evaluation at the frontier can be built and enforced.

Voluntary frameworks do not become binding ones automatically. They become binding when evaluation capacity and enforcement authority are built before the political will to build them runs out.

What to do with this

For governance architects: the diagnostic test for any AI governance instrument is whether it creates inspection authority the regulated party cannot veto and reserves a meaningful enforcement backstop. Frameworks that lack both produce records of commitment rather than actual governance.

For policy thinkers: the EU AI Office’s oversight of general-purpose AI models is the live test of binding evaluation at the frontier. Its enforcement powers come into force this August, and the outcome will shape whether similar frameworks spread globally. It deserves more serious attention than it is currently receiving.

For AI leaders: voluntary commitments made without audit rights will eventually be replaced by binding rules. Organizations that demonstrate third-party-verified safety evaluations now (not just published responsible scaling policies but assessments an independent reviewer has actually tested) will have shaped what those binding rules look like. Those that do not will be governed by whatever emerges when the political will for mandatory standards arrives.

The technical work on alignment is necessary. It is not sufficient, and it is downstream of governance choices that engineers alone do not have the standing to settle. That architecture is being built right now. The question is who is in the room.

Endnotes

1. “OpenAI completes its for-profit recapitalization,” TechCrunch, October 28, 2025, https://techcrunch.com/2025/10/28/openai-completes-its-for-profit-recapitalization/.
2. Delaware Department of Justice, “AG Jennings completes review of OpenAI recapitalization,” State of Delaware News, October 28, 2025, https://news.delaware.gov/2025/10/28/ag-jennings-completes-review-of-openai-recapitalization/.
3. Anthropic, “Claude’s Constitution,” Anthropic, May 9, 2023, https://www.anthropic.com/news/claudes-constitution.
4. Anthropic, “Collective Constitutional AI: Aligning a Language Model with Public Input,” Anthropic, October 17, 2023, https://www.anthropic.com/research/collective-constitutional-ai-aligning-a-language-model-with-public-input.
5. Department for Science, Innovation and Technology (UK), “Initial £100 million for expert taskforce to help UK build and adopt next generation of safe AI,” GOV.UK, April 24, 2023, https://www.gov.uk/government/news/initial-100-million-for-expert-taskforce-to-help-uk-build-and-adopt-next-generation-of-safe-ai.
6. Alphabet Inc., “Third Quarter 2025 Earnings Results (Exhibit 99.1),” U.S. Securities and Exchange Commission, October 2025, https://www.sec.gov/Archives/edgar/data/0001652044/000165204425000087/googexhibit991q32025.htm.
7. Department for Science, Innovation and Technology (UK), “Frontier AI Safety Commitments, AI Seoul Summit 2024,” GOV.UK, updated February 7, 2025, https://www.gov.uk/government/publications/frontier-ai-safety-commitments-ai-seoul-summit-2024/frontier-ai-safety-commitments-ai-seoul-summit-2024.
8. US Department of Transportation Office of Inspector General, “FAA’s Oversight of Boeing’s 737 MAX,” Report No. AV-2021-020, February 23, 2021, https://www.oig.dot.gov/sites/default/files/FAA%20Certification%20of%20737%20MAX%20Boeing%20II%20Final%20Report%5E2-23-2021.pdf.
9. Iason Gabriel, “Artificial Intelligence, Values and Alignment,” Minds and Machines 30 (2020), https://arxiv.org/abs/2001.09768.
10. US Food and Drug Administration, “Sulfanilamide Disaster,” FDA, accessed June 10, 2026, https://www.fda.gov/about-fda/histories-product-regulation/sulfanilamide-disaster.
11. International Atomic Energy Agency, “The Nuclear Non-Proliferation Treaty (NPT),” IAEA, accessed June 10, 2026, https://history.state.gov/milestones/1961-1968/npt.
12. European Parliament and Council, Regulation (EU) 2024/1689 (AI Act), Official Journal of the European Union, June 13, 2024, https://eur-lex.europa.eu/eli/reg/2024/1689/oj.
13. Council of the European Union, “Artificial intelligence: Council and Parliament agree to simplify and streamline rules,” Consilium, May 7, 2026, https://www.consilium.europa.eu/en/press/press-releases/2026/05/07/artificial-intelligence-council-and-parliament-agree-to-simplify-and-streamline-rules/.
14. EU Artificial Intelligence Act (editorial), “Enforcement of Chapter V under the EU AI Act,” EU AI Act, accessed June 10, 2026, https://artificialintelligenceact.eu/enforcement-of-chapter-v-under-the-eu-ai-act/.