Most of the current debate around artificial intelligence focuses on capability and accuracy. Is the algorithm biased? Is the model reliable? Can we explain its outputs? These are important questions. They are not the most important ones. History suggests the central challenge may be legitimacy.

People routinely accept flawed decisions from institutions they trust. They reject technically correct decisions from institutions they don’t. We defer to judges whose rulings we disagree with, to doctors whose recommendations we challenge, to governments whose policies we oppose. That deference does not reflect a belief in infallibility. It reflects a recognition that the decision-maker holds legitimate authority.

As AI systems take on decision-making roles in hiring, lending, content moderation and other specialities, we are discovering that accuracy alone does not produce that recognition.

What Makes Authority Legitimate?

Max Weber, writing in the early twentieth century, identified three sources of legitimate authority: traditional authority rooted in long-standing custom; charismatic authority derived from the personal qualities of a leader; and rational-legal authority grounded in rules, procedures, and institutional oversight. Modern democratic governance relies primarily on the third type. It is impersonal. It does not depend on who makes the decision. It depends on the system of accountability surrounding the decision-maker.

AI systems face a legitimacy deficit across each of these dimensions. They carry no tradition. They have no charisma. The rational-legal frameworks that would confer authority have not been built around them.

Where This Plays Out

The deployment of predictive policing tools in several American cities offers an instructive case.[2] Many of these systems were technically defensible. Community resistance focused on accountability as much as — and in some cases more than — accuracy: who built the model, what data trained it, who could challenge the outputs, and who bore responsibility when those outputs contributed to harm.

Automated resume screening systems have encountered similar scrutiny. The critique arrived from two directions. Performance failures, including documented bias against women, older candidates, and racial minorities in several widely studied systems, were one driver. Process failures were another: no one could explain, contest, or appeal the outputs. Neither was reducible to the other.

Governance Matters More Than Accuracy

This points toward a reframing. Legitimacy is not a property of the AI system itself; it is a property of the governance structures surrounding it.

The relevant questions are less about technical performance and more about institutional design:

• Who deployed the system, and under what mandate?
• Who provides oversight, and with what authority?
• What appeal mechanisms exist for affected individuals?
• Who is accountable when the system fails?

An AI system embedded in strong oversight structures, with meaningful appeal rights and clear lines of accountability, can carry legitimacy. The same system operating as an opaque mechanism, answerable to no one, cannot, regardless of its accuracy.

A Contrarian Argument Worth Taking Seriously

The dominant assumption in the technology industry is that better AI drives greater adoption. As systems become more capable, resistance will fade. That assumption deserves scrutiny.

Better AI deployed without corresponding legitimacy structures may actually increase resistance. As systems grow more capable, the decisions they influence grow in significance. Higher stakes make accountability more important, not less. A system that was tolerable when it ranked resumes becomes intolerable when it influences prison sentences or determines insurance eligibility.

If capability advances faster than governance, the legitimacy gap widens precisely when it matters most.

What This Means

The defining challenge of AI governance in the years ahead will not be whether machines can make sound decisions. In many domains, they already can. The challenge will be whether humans will accept those decisions as legitimate.

Acceptance will not come from more accurate models; it will come from building the institutional infrastructure that surrounds them: oversight bodies, appeal mechanisms, accountability frameworks, and the democratic processes that define what these systems may and may not decide. Societies built that infrastructure for courts, financial institutions, and regulatory agencies. It took generations and remains imperfect. The work of building equivalent structures for AI has barely begun.

The history of technology includes many capable systems that failed to achieve broad adoption because the social and institutional architecture to support them was never constructed. AI governance risks repeating that pattern, albeit at greater cost and at greater scale.

Getting the governance right is not secondary to the technical work. For AI systems that make or shape consequential decisions, it is the work.