Research PaperLiving document, v1.1

What Constitutes Meaningful Oversight: Determining Accountability for AI Across Regions, Industries, and Functions

Every serious AI regime now requires a human to oversee the system. Almost none of them say what that human has to be able to do. Here is a testable bar, and what it exposes.

Rahul Jindal|50 min read|First published 2026-08-04|Last verified 2026-08-04
Abstract

Every serious AI regulation in force in 2026 requires that a human oversee the system. Almost none of them specify what that human must be able to do, and none requires anyone to check whether the oversight changed an outcome. A survey of 41 such policies found that not one defines what would make oversight meaningful. Where researchers have measured it, the overseer underperforms the tool, cannot evaluate the tool, cannot evaluate themselves, and grows more confident as they get worse.

This paper does three things. It sets a testable bar for oversight in five questions, each grounded in a measured finding rather than a principle: Name, Sight, Power, Time, Proof. It separates oversight from accountability and proposes an allocation rule for who answers for what across a value chain that now runs from a foundation model through a fine-tune and an integrator to a deployer and a professional user. And it maps how the answer changes across regions, industries and functions, because it does change, and the differences are more instructive than the similarities.

The findings are uncomfortable in both directions. The EU's human oversight article is not in force and was deferred to December 2027 because the standards to assess against did not exist. The US banking agencies rescinded the fifteen-year-old model risk framework the rest of the economy is told to copy, and carved generative and agentic AI out of its successor. Colorado repealed the only US duty of care on AI developers before it took effect. The world's first algorithmic bias audit mandate produced audits from 4.6% of the employers checked. And a sepsis model deployed at hundreds of hospitals missed two thirds of the disease it was built to catch, for years, because approval and oversight are different things and it needed neither.

What works is borrowed, cheap, and mostly absent from AI law: named individual certification with personal exposure, public and named external audit, a bounded-immunity reporting channel held by someone with no power to punish, severance of investigation from liability, structured incident reporting with public data, and post-market surveillance. Each has a measured track record in another industry. The paper closes with a 90-day build for operators, nine design rules for regulators, and seven questions for a board.

This paper is maintained, not archived

Update this

AI regulation moves faster than publishing cycles. Half the dated claims below have a shelf life measured in weeks. Press this and a refresh job gets filed: every time-stamped claim goes back to its primary source, anything that moved gets rewritten, new statutes, rulings and enforcement actions since 2026-08-04 get folded in, and the paper republishes with a new version number and a changelog line saying what changed. You are reading v1.1.

00

Executive summary

Ten findings, and what to do about them. The rest of the paper is the evidence.

  1. 01Every serious AI regime requires human oversight. None defines it. A survey of 41 policies mandating oversight of government algorithms found that not one gives a standard for telling whether a particular arrangement is meaningful. The requirement is universal and unfalsifiable at the same time.
  2. 02Where it has been measured, the overseer underperforms the tool. In the cleanest experiment, the human-plus-algorithm team scored 2.6% worse and had a 46.5% higher false positive rate than the algorithm alone, and only 23.7% of overseers beat the tool. Confidence ran negative to performance.
  3. 03The standard remedies are contradicted by the evidence. Training and instruction do not prevent automation bias. Explanations increase acceptance of wrong answers. Expert scepticism about AI provenance does not translate into resistance to bad advice.
  4. 04Article 14 of the EU AI Act is not in force. The Digital Omnibus, in force 27 July 2026, moved standalone high-risk obligations to 2 December 2027. The proximate cause was a standards failure: zero harmonised standards had been cited in the Official Journal, so no provider could obtain a presumption of conformity.
  5. 05The sector that invented model accountability withdrew it from AI. US banking guidance revised on 17 April 2026 rescinds SR 11-7, expressly excludes generative and agentic AI from scope, and disclaims its own enforceability.
  6. 06The only US statute that ever imposed a duty of care on AI developers was repealed before it took effect. Colorado's AI Act was delayed twice and replaced in May 2026 with a notice-and-explanation statute.
  7. 07Mandated bias audits produced almost no audits. Of 391 New York City employers checked, 4.6% posted the required report. In two years the enforcement agency received two complaints and issued two demand letters. In one 32-company review the regulator found one problem and state auditors found seventeen.
  8. 08Approval is not oversight. A sepsis model deployed at hundreds of US hospitals missed 67% of sepsis cases and alerted on 18% of all admissions. It was never approved because it never needed to be, and the only reason anyone knows is that one health system chose to look.
  9. 09No jurisdiction has an agent-specific accountability regime. The only binding agent-specific obligation anywhere in law is a disclosure duty. The party holding the controls that would have prevented every documented agentic incident, the integrator, is a regulated role nowhere.
  10. 10What works is borrowed, and cheap. Named individual certification with personal exposure. Public, named external audit. Bounded-immunity reporting held by a non-regulator. Severance of investigation from liability. Structured incident reporting with public data. Post-market surveillance. All six have measured track records in other industries. None is in AI law.

If you read one more section, read 03 for the test and 12 for the build. If you run a function, read 08, which sorts HR, Finance, Marketing, Operations and the rest by what actually bites each one. If you are a regulator, read 10 and 13. If you are on a board, read 14.

01

The word that stopped meaning anything

Every serious AI regime on earth requires human oversight. Almost none of them say what the human has to be able to do.

My job at Google is AI transformation, so I spend my weeks inside the gap between what a governance document says and what an organisation actually does. The gap is widest around one word. Oversight.

Read the rules and you would think the question is settled. The EU AI Act says high-risk systems must be designed so they can be "effectively overseen by natural persons." India's governance guidelines put human oversight in the second of seven principles. Korea's Basic Act requires it for high-impact AI. Australia's guidance for adopters lists "maintain human control" as one of six essential practices. The UAE Charter has a principle for it. Almost every enterprise AI policy I have read has a slide about it.

Now try to fail one of those tests. Try to design a system that a regulator would say is not overseen. It is surprisingly hard. Put a person in front of a screen, give them a button, write a procedure, and you have satisfied the text of nearly every oversight requirement in force today. Whether that person can read the output, has time to judge it, is permitted to say no, or suffers anything at all when they wave through a bad one, is left open.

That last point is the one I keep coming back to. If you ran any other control this way you would be laughed out of the room. You would not sign off a fire suppression system that had never been tested. But a human oversight function that has never once overturned the model is treated as evidence that the model is good, when the honest reading is that the oversight is not working.

Three questions, and why they are the whole subject

Strip away the frameworks and every accountability regime ever built answers three questions about a bad outcome. Who knew. Who could have stopped it. Who pays. Aviation answers them with an independent investigator, a mandatory reporting system, and a manufacturer on the hook for a certified design. Medicine answers them with a licensed professional, an approval file, and malpractice liability. Banking answers them with an independent validation function, a named senior manager, and a regulator that can remove them.

AI governance in 2026 answers the first question partially, the second badly, and the third almost not at all. This paper is an attempt to fix that in a way an operator can use on a Monday and a regulator could write into an instrument.

What this paper does

  1. 01Sets a testable bar for oversight. Five conditions, each of which can be checked in an afternoon against a real system, and each of which is grounded in evidence about how human oversight actually fails.
  2. 02Separates oversight from accountability. They are different problems. A system can be well overseen and still leave nobody accountable, and a system with no oversight at all can have crisp accountability. Confusing them is why so many regimes produce neither.
  3. 03Proposes an allocation rule for who answers for what across a value chain that now runs from a foundation model to a fine-tune to an integrator to a deployer to a professional user, with third-party agents wired in that nobody in the chain wrote.
  4. 04Works the variance. The right answer differs by region, by industry, and by function, and the paper maps all three rather than pretending one model travels.
  5. 05Ends with what to actually do: a 90-day build for an operator, design rules for a regulator, and the questions a board should be asking.

One thing this paper is not. It is not an argument that more regulation is the answer, or that less is. The evidence in it cuts both ways and I have tried not to flatten that. The argument is narrower and I think harder to dodge: whatever quantity of oversight a jurisdiction or a company chooses, most of what is currently labelled oversight would not survive contact with the five questions in section 03, and that is a problem for people who want more rules and people who want fewer.

02

What the evidence says about human oversight

This is the part of the subject that policy has not read. Forty years of human factors research, and it is not encouraging.

In 1983 Lisanne Bainbridge published a five-page paper in Automatica about what happens when you automate an industrial process. She was not writing about AI. She was writing about control rooms. Her argument is the reason every human oversight requirement written since should be read with suspicion, and it is a logical argument rather than an empirical one, which is why forty-three years of better interfaces have not dissolved it.

If the computer is being used to make the decisions because human judgement and intuitive reasoning are not adequate in this context, then which of the decisions is to be accepted? The human monitor has been given an impossible task.

Read it twice. If you adopted the system because it beats the human at the task, you cannot then appoint that human as the arbiter of whether it got the task right. Bainbridge also gave the number that every oversight rota quietly violates: it is impossible for even a highly motivated person to maintain effective attention on a source where very little happens for more than about half an hour. And she named the recursion nobody solves. If you fix the attention problem with an automatic alarm, who notices when the alarm system stops working.

Barry Strauch revisited the argument for IEEE in 2017 and found the ironies recurring in accident investigations rather than resolved by design. That is the state of the field. We have known this since before most people writing AI policy were born.

The survey that should have ended the debate

Ben Green surveyed 41 policy documents that mandate or guide human oversight of government algorithms. Legislation, government guidance, and the operating manuals and court cases for two real risk assessment tools. His conclusion is the single most important citation in this paper.

Rather than protect against the potential harms of algorithmic decision-making in government, human oversight policies provide a false sense of security in adopting algorithms and enable vendors and agencies to shirk accountability for algorithmic harms.

Green found that none of the 41 policies defines what would make oversight meaningful. They all agree that a human rubber-stamping decisions is not enough. Not one gives a standard for telling whether a particular arrangement clears the bar. He also names the loophole that follows: because any nominal human involvement removes a decision from the "solely automated" category, rules like GDPR Article 22 create an active incentive to install superficial review as a compliance device.

Then somebody measured it

Green and Chen ran the experiment. 554 people made 13,850 predictions about defendant risk, half of them seeing a risk assessment score and half not. The score helped the humans. It did not help enough.

2.6%
worse average reward for the human-plus-algorithm team than the algorithm alone
Green and Chen, FAT* 2019, p < 10 to the minus 8
46.5%
higher false positive rate for the team than for the algorithm alone
Same study
23.7%
of overseers beat the tool they were overseeing. 64.1% did worse.
195 of 304 participants underperformed the model

Two further results from the same paper matter more than the headline. Confidence was negatively correlated with performance. The more sure a participant was, the worse they did. And their assessment of the tool's accuracy had no significant relationship with the tool's actual accuracy. So the overseer could not evaluate the system, and could not evaluate themselves either.

In a second study across 2,140 participants, Green and Chen found the risk assessment silently reweighted how people decided. In the pretrial setting it made them more sensitive to perceived risk and increased the racial disparity in detention by 1.9%. In the loans setting it made them more risk averse and cut government aid by 8.3%. Self-reported confidence did not move in any of the four conditions. The behavioural shift was invisible to the people undergoing it.

The remedies that do not work

Three interventions appear in nearly every AI governance document. All three are contradicted by the literature.

Exhibit 1: standard remedies against the evidence
The remedy in the policyWhat the research foundSource
Train the reviewers, and instruct them to verifyAutomation bias "cannot be prevented by training or instructions" and complacency "cannot be overcome with simple practice." It occurs in naive and expert participants alike, and in teams as well as individuals. The root cause is attentional allocation under load, so remedies aimed at knowledge or motivation are aimed at the wrong variable.Parasuraman and Manzey, Human Factors 52(3), 2010
Give the human an explanation so they can judge the outputExplanations "increased the chance that humans will accept the AI's recommendation, regardless of its correctness." Explanations with no basis in the model's actual functioning still increased trust. Transparency reduced people's ability to detect and correct model errors.Bansal et al., CHI 2021; Lai and Tan 2019; Poursabzi-Sangdeh et al. 2021
Use experienced professionals who will be appropriately scepticalRadiologists rated advice as lower quality when told it came from an AI, and their diagnostic accuracy was still significantly worse when the advice was wrong, whatever its purported source. Attitudinal distrust is not behavioural resistance.Gaube et al., npj Digital Medicine, 2021

The clinical effect sizes are the cleanest in the literature. Lyell et al. (2017) found incorrect decision support increased omission errors by 24.5% to 33.3%, and between 51.7% and 65.8% of clinicians actively accepted a false positive alert.

Two things do work, and neither is in any binding instrument. Cognitive forcing functions, which interrupt the reviewer and make them commit before seeing the model's answer, measurably reduce overreliance. And calibrated uncertainty, telling the reviewer how confident the system is, made participants slow down and think analytically. Note the catch on the first one. In Buçinca and colleagues' study, people gave the least favourable ratings to the designs that reduced overreliance the most. An intervention users dislike will not survive a product-led design process. It has to be mandated to exist.

The honest counter-evidence

This literature is not unanimous and a paper that pretends otherwise is doing the same thing it accuses regulators of. Alon-Barkat and Busuioc ran three pre-registered Dutch experiments across 605, 904 and 1,345 participants, the last group actual civil servants, and found no evidence for automation bias. What they found instead was selective adherence: people followed advice more readily when it matched a group stereotype, with no significant difference between algorithmic and human-expert advice. De Arteaga and colleagues found that staff using the Allegheny Family Screening Tool did override many algorithmic errors.

If that reconciliation is right, the problem is not deference to machines. It is deference to advice, plus stereotype confirmation, which changes the remedy from AI literacy to advice handling and bias control. Worth noting that the third Dutch study ran shortly after the childcare benefits scandal, and the authors attribute the absence of selective adherence to bureaucrats' heightened awareness in its aftermath. A public scandal measurably changed reviewer behaviour. That is evidence for building an accountability ecosystem rather than for trusting an individual overseer.

So the defensible claim is narrower than the polemical one, and I want to state it precisely. Human oversight is not uniformly useless. It is unreliable, context-dependent, and almost never measured, and no policy anywhere currently requires it to be measured. That is the finding that the rest of this paper builds on.

Who ends up carrying it

Madeleine Clare Elish gave the failure mode its name. A moral crumple zone: the human in a complex automated system becomes the component that absorbs the moral and legal impact when the system fails, protecting the integrity of the technology at the expense of the nearest person.

The report even cites Parasuraman and Manzey by name and holds that Uber failed to design against a risk the published science had already established. That is the accountability move AI regulation has not made: treating known human factors science as a design obligation whose breach belongs to the organisation. Everyone quotes Tempe as a story about a distracted driver. It is a story about a company that designed a one-second silence into an emergency and then wrote the human into the gap.

03

The five tests

Name, Sight, Power, Time, Proof. If you cannot answer all five about a system running in your organisation today, you do not have oversight. You have a person.

Everything in section 02 says that the current bar is unfalsifiable. Here is a bar you can fail. Five questions, each of which can be answered in an afternoon against a real system, each of which is grounded in a specific finding rather than in a principle.

Exhibit 2: the five tests, and what each is really asking
TestThe questionGrounded in
NameIs there a named individual who answers for this, and does their exposure exceed the front-line reviewer's? A committee is not a name. A policy is not a name.Elish's moral crumple zone. Green's second flaw, that oversight shifts accountability from the leaders who chose the system to the operators who are powerless. The working counter-model is 18 U.S.C. 1350, where a CEO signs personally.
SightCan the overseer reach a different conclusion using evidence the system did not hand them? If the only input is the model's verdict plus its rationale, that is review, not oversight.Bansal et al: explanations increase acceptance regardless of correctness. Goddard et al identify "the provision of information versus recommendation" as a mitigator. Give them evidence, not a verdict.
PowerCan they stop it without asking permission, and does saying no cost them nothing? An override that requires a manager's sign-off is a request, not a control.EU AI Act Art 14(4)(e) requires a stop button and says nothing about who may press it or what happens to them afterwards. The Tempe operator had authority and 1.2 seconds.
TimeDo they have the seconds, the attention budget and the base rate to actually judge? What else is this person doing, and what redundancy was removed to pay for the deployment?Bainbridge's half-hour vigilance limit. Parasuraman and Manzey on complacency under multi-task load. The NTSB on Uber removing the second operator.
ProofHas anyone measured whether the oversight changes outcomes, comparing overseer-plus-system against system-alone? Is the override rate instrumented and reported?Green and Chen: the team was 2.6% worse than the model alone and confidence ran negative to performance. Self-report is invalid. Mosier et al: overseers confabulate having checked.

One regulator has now written something close to this down, which is worth noting because it means the position is not idiosyncratic. The Financial Stability Board's June 2026 sound practices define the bar directly.

Oversight is meaningful only where humans have sufficient ability, authority, and incentive to intervene, as opposed to oversight that is nominal or driven by tick-the-box compliance.

Ability, authority and incentive. That is Sight, Power and the consequence half of Name. What the FSB formulation leaves out is Time, which is the condition agents break, and Proof, which is the condition that makes the other four checkable rather than assertable.

And Proof has now been written down by a regulator too, in the least likely place: the FDA's guidance on using AI in drug development. It asks sponsors to evaluate the performance of the human-AI team, rather than the performance of the model in isolation, inside a matrix of model influence against decision consequence where bringing genuinely independent evidence lowers your burden. That is the correct incentive in one design move. A real check pays for itself. A rubber stamp buys nothing.

Scoring it

Score each test 0, 1 or 2. Zero if the answer is no or unknown. One if it is partially true or true on paper. Two if you could show a sceptical outsider the evidence today.

Exhibit 3: what the score means
ScoreReadingWhat to do
0 to 3Oversight theater. The human is present to satisfy a clause and to absorb blame.Either fix it or stop claiming oversight. The second option is more honest and cheaper. If the system cannot be defended without the oversight claim, do not deploy it.
4 to 6Partial. Usually Name and Power are present and Sight, Time and Proof are missing. This is the most common enterprise state.Instrument first. Proof is the cheapest test to move and the one that tells you which of the others is really broken.
7 to 8Real oversight on a specific system. Rare.Check that it survives scale. Most arrangements that pass at ten decisions a day fail at a thousand.
9 to 10The arrangement is doing work and you can prove it.Publish the method. Almost nobody has one and the field needs the example.

Applying it to the instruments in force

The tests are more useful pointed at regulation than at a single deployment, because they show what each regime forgot.

Exhibit 4: the five tests applied to major instruments
InstrumentNameSightPowerTimeProofTotal
EU AI Act Arts 14 and 26 (from Dec 2027)0. Duties attach to legal persons only. The assigned natural person signs nothing.1. Art 13 requires instructions for use, but the overseer's channel is the system's own output.2. Art 14(4)(d) and (e) give an explicit right to disregard, override and halt.1. Competence, training and support required. No load, latency or staffing standard.0. Nothing measured, nothing reported, no override rate anywhere.4 / 10
UK Senior Managers regime applied to AI2. A named senior manager with a statement of responsibilities and personal sanction.0. Nothing AI-specific about the information channel.1. Implied by the responsibility, not specified.0. Not addressed.1. Supervisory examination, but no published oversight metric.4 / 10
SEBI Regulation 16C (India)2. The regulated entity is solely and non-delegably responsible for AI outputs, built or bought.0. Not addressed.0. Not addressed.0. Not addressed.0. Not addressed.2 / 10, and it may still be the most enforceable rule in this table
Colorado, as replaced by SB 26-1890. The duty of reasonable care was repealed before it took effect.1. Adverse-outcome explanation within 30 days.1. Meaningful human review is required and undefined.0.0.2 / 10
California SB 531. Corporate rather than personal, though the whistleblower protection does reach individuals.1. Published frameworks and transparency reports.0. Not an oversight statute.0.1. Critical safety incident reporting in 15 days, or 24 hours where death is imminent.3 / 10, and it is aimed at a different problem

The pattern in this table is the paper's finding. Regimes score on Power because a stop button is easy to legislate. They score zero on Proof because measuring whether oversight works would expose that it often does not. Nobody scores on Name except the two regimes that borrowed the idea from financial services.

Notice something about the highest possible score anywhere in that table. Four out of ten. Not one instrument in force in any major jurisdiction would pass its own oversight requirement if the requirement had teeth.

04

The oversight ladder

Six levels. Almost every regulation targets level 3, which the evidence says is the weakest rung on the ladder.

Oversight is not binary and treating it as binary is how regimes end up mandating the version that does not work. Here are six levels, ordered by how much they actually constrain the system, with the honest note on each about what the research says.

Exhibit 5: the oversight ladder
LevelWhat it meansHonest assessment
L0 · NoneThe system acts. Nobody is told.Appropriate for genuinely low-consequence, high-volume decisions. Pretending otherwise wastes the oversight budget you need elsewhere.
L1 · NotifiedA human is told after the fact.Cheap and better than nothing, because it creates a record. It is not a control. Nothing about being told changes an outcome that already happened.
L2 · Reviewable on requestThe affected person can ask for a human to look again.This is GDPR Article 22's remedy and Green names its flaw: it puts the burden on the individual to request review after they have been harmed, and most people have neither the means nor the knowledge to use it.
L3 · Human in the loopA human approves each output before it takes effect.The level almost every regulation mandates, and the one section 02 is about. It fails on Bainbridge's logic, produces automation bias, is defeated by explanations, cannot be trained out, and creates a moral crumple zone. It is not worthless. It is the weakest rung and it is the one being legislated.
L4 · Human in commandThe human sets the policy and the constraints. The system executes inside them. The human can change the policy mid-flight and the system demonstrably responds.This is Santoni de Sio and van den Hoven's tracking condition, and it is the right target for anything running at machine speed. It moves the human from adjudicating outputs, which they cannot do, to setting bounds, which they can. Nothing in binding law requires it.
L5 · InstitutionalAn independent body can compel information, inspect, publish findings, and stop the deployment. A named individual attests that the controls exist and work.Green's own remedy: shift from human oversight to institutional oversight, with the deploying organisation required to justify both the system and its proposed oversight to a body that can say no. This is the level that has a track record, and it comes from other industries.

Choosing a level

Green's allocation table is the most usable instrument I have found for this and it deserves to be better known. He puts decisions on two axes: how much discretion the decision needs, and how trustworthy the algorithm actually is. The asymmetry is the point.

Exhibit 6: after Green's allocation table, Computer Law and Security Review 45 (2022), Table 1
Low need for discretionHigh need for discretion
Low algorithm trustworthinessPrimarily or solely human. Algorithms only to the extent rigorous research shows a benefit.Solely human decision-making.
High algorithm trustworthinessPrimarily or solely algorithmic.Primarily or solely human. Algorithms only to the extent rigorous research shows a benefit.

Look at what is absent. There is no cell anywhere in this table that reads "algorithm decides, human oversees." That configuration, which is what almost every AI deployment in the world currently is, does not appear in the scheme at all. Green's position is that the configuration is an evasion rather than a design.

I do not fully agree with him, and the disagreement is worth stating because it changes what an operator should do on Monday. Green is writing about government algorithms, where the affected person has no exit and the legitimacy cost of a wrong decision is very high. In commercial settings with reversible decisions, cheap appeals and real competitive pressure, an L3 arrangement with instrumented override rates can be a reasonable stage on the way to L4. What is not reasonable is treating L3 as the destination, which is what the AI Act, Colorado's replacement statute and most enterprise policies currently do.

05

Determining accountability: an allocation rule

Oversight asks whether anyone is watching. Accountability asks who answers. They are different problems and most regimes only solve the first.

In 1996 Helen Nissenbaum wrote that computerisation erodes accountability through four barriers, and thirty years on they read like a description of the foundation model supply chain. Many hands, so no one act causes the harm. Bugs, treated as inevitable and therefore excusable. The computer as scapegoat. And ownership without liability, an industry claiming property rights over software while disclaiming responsibility for what it does.

Code reuse, taken as a virtue in software development, has now been extended to model reuse.

That sentence was written in 2022 and it states the whole problem in advance. The same paper gives the definition I use for the rest of this section, and it is the sharpest one in the literature: accountability exists when an accountable actor has an enforceable obligation to a forum to explain and justify itself. Two words carry the weight. Enforceable, and forum. Most of what the industry calls AI accountability in 2026 fails on both. A model card is an explanation offered to nobody in particular, with no consequence for its contents.

Four gaps, four different fixes

Filippo Santoni de Sio and Giulio Mecacci made the most policy-useful move in this literature by refusing to treat the responsibility gap as one thing. They split it into four, and the reason it matters is that the four have different remedies.

Exhibit 7: the four responsibility gaps and what closes each
GapWhat is missingWhat actually closes it
CulpabilityNobody is blameworthy for the harmA liability rule. Strict, or fault with a burden shift.
Moral accountabilityNobody can explain or justify what happenedLogging, traceability and a disclosure duty.
Public accountabilityNo institution owes the public an accountA regulator with an incident register and the power to compel.
Active responsibilityNobody holds the forward-looking duty to prevent recurrenceA named, resourced human role. Nothing else closes it.

After Santoni de Sio and Mecacci, Four Responsibility Gaps with Artificial Intelligence, Philosophy and Technology (2021). The fourth row is the one AI law has never built. Disclosure statutes close the second gap and are frequently sold as closing the fourth.

The same authors, with Jeroen van den Hoven, also gave us the only operational definition of meaningful human control I have found that can actually be failed. It has two conditions. Tracking: the system must be demonstrably and verifiably responsive to the human moral reasons relevant in the circumstances. Tracing: its actions must be traceable to a proper moral understanding on the part of one or more relevant humans, which requires that someone understands what the system can do, and knows that they may be held accountable for it.

Who is even in the chain

Before allocating, it helps to see how many parties exist and how few of them any regime can see. A 2026 enterprise deployment routinely involves eleven distinguishable actors. Most statutes recognise two.

Exhibit 8: the value chain, and who regulates each link
PartyWhat only they controlRegulated as
Foundation model developerPretraining data, weights, base capability, refusal behaviourEU: GPAI provider (Art 53). California SB 53: large frontier developer.
Fine-tunerTask capability, and how much safety tuning survivedEU: becomes a provider above a compute ratio. Elsewhere: invisible.
Application developerPrompts, scaffolding, tool grants, interfaceEU: provider of an AI system. US states: developer.
Systems integratorThe wiring from model to enterprise data and toolsNowhere. Not a regulated role in any regime.
Deployer / operatorUse case, input data, who staffs oversightEU Art 26, Colorado, Texas, Korea.
Professional userThe actual decisionSectoral law and malpractice, not AI law.
End userConsent, prompt contentTreated as a protected person, not a duty bearer.
Data providerCorpus provenanceCopyright and data protection only.
Compute providerWho gets to train whatReporting proposed in the US, never enacted federally.
Distributor / importerMarket placement in a jurisdictionEU Arts 23-24, flipping to provider under Art 25.
Auditor / evaluatorIndependent assuranceEU notified bodies for biometrics only. Otherwise contractual.

The integrator row is the important one. The party that grants an agent its OAuth token, wires it to a third-party tool server nobody in the chain wrote, and decides whether it can write to production, is a regulated role in no jurisdiction on earth.

How the major regimes allocate, and where each breaks

Exhibit 9: allocation logic by regime, with the failure mode
RegimeAllocation logicWhere it breaks
EU AI ActProvider vs deployer, with Art 25 flipping the label onto anyone who rebrands, substantially modifies, or repurposes a system into high risk. Art 53(1)(b) pipes documentation downstream."Substantial modification" is undefined for models that change continuously. Art 25(2) lets an upstream provider disclaim its cooperation duty by writing the right sentence into its terms. And the instrument that was meant to allocate civil liability, the AI Liability Directive, was withdrawn in February 2025.
ColoradoWas the only US statute to impose a duty of reasonable care on developers and deployers against algorithmic discrimination.Repealed before it ever took effect. SB 26-189, signed 14 May 2026, replaced it with a notice-and-explanation statute effective 1 January 2027. The care duty is gone. Discrimination liability reverts to generic state law.
Texas TRAIGADuties on developers and deployers, with prohibited conduct pinned to intent.An intent standard is close to a nullity for statistical harm. Nobody forms an intention about a disparate error rate. No private right of action, a 60-day cure period, and enforcement resting entirely on one Attorney General's discretion.
ChinaOne party: the service provider offering to the public inside China. Content, labelling, filing and user management all land there.Clean and genuinely enforced, but it is content allocation rather than risk allocation. An enterprise wiring a domestic model into an internal workflow is largely outside the frame, and there is no civil route for an injured individual.
KoreaThe AI business operator, plus a mandatory domestic representative above revenue and user thresholds.The representative is a service-of-process device, not a fiduciary one. The maximum fine for failing to tell users they are talking to an AI is roughly USD 21,000, against a revenue trigger of roughly USD 662m.
IndiaThe intermediary, via conditionality on Section 79 safe harbour. Separately, SEBI Regulation 16C puts strict, non-delegable responsibility for AI outputs on the regulated entity, built or bought.The intermediary route reaches platforms and not enterprise deployment. Regulation 16C reaches only SEBI registrants. DPDP has no Article 22 analogue, so an individual has no claim against an automated decision as such.

The rule

Five principles compete for how to allocate loss, and each is right about something and wrong alone. Least-cost avoider is the correct instinct but the cheapest avoider differs per failure mode and cannot be identified from outside. Control is clean but agentic systems distribute control across parties who each hold a necessary and insufficient condition. Benefit correctly implicates model developers who monetise every downstream deployment, and is a poor deterrent because the benefit holder is often not the risk creator. Capacity to insure is practically decisive and currently pointing the wrong way, because carriers are excluding rather than pooling.

The fifth is the strongest for AI specifically, and it is Catherine Sharkey's information-forcing argument. Put the duty on whoever knows what nobody else can learn. That gives us a rule.

Exhibit 10: the allocation, layer by layer
LayerOwesOn what basis
Foundation model developerDangerous-capability evaluation results, training data provenance at class level, known failure modes, refusal boundaries, and how much safety tuning survives fine-tuning. Strict liability for weight-level defects only.Information asymmetry plus benefit. Nobody else can measure a base model's latent capability.
Fine-tunerDisclosure of which safety properties the fine-tune degraded.Control. The documentation duty should trigger at a far lower compute ratio than the full provider duty.
Application developer and integratorThe blast-radius controls. No write access without confirmation. No lethal-trifecta combination in one session. Tool scopes minimised and elevated incrementally. Every action logged with an instance ID.Least-cost avoider, decisively. This is the highest-leverage duty in the chain and it currently sits with the only unregulated party.
DeployerUse-case appropriateness, input data suitability, oversight staffed with real competence and real authority to stop, and incident reporting.Control plus benefit. Roughly EU Article 26, which is broadly right.
Professional userThe ordinary professional standard, unmodified. AI does not lower it.Licensure. But the learned intermediary doctrine should be narrowed: it should protect a vendor only where the professional was actually given enough to exercise independent judgement.
Compute provider, distributorVisibility duties only. Know your training customer above a threshold, preserve records.They cannot evaluate what they host. Do not give them liability they cannot discharge.
Auditor and evaluatorAn independent duty to the public, with a safe harbour and a publication right.Without a publication right an evaluator is not a forum. It is an audience.

Test it against real incidents, because a rule that cannot be applied to a case is a slogan. When Replit's coding agent deleted a production database in July 2025, the agent had unconfirmed write access to production. That is an integrator failure and the rule lands there, not on the model developer. When Brave demonstrated that hidden text in a web page could make Perplexity's Comet browser read a user's email, fetch a one-time password and post both to Reddit, the vendor had combined all three legs of a publicly known attack class in one session. Integrator again. When Anthropic reported disrupting a state-linked espionage campaign that used Claude Code for most of its work, the attacker was the application layer, liability lands on the human operators, and the residual question about the developer's access controls is an ordinary negligence question with an answer.

Three cases, three identifiable defendants, and not once did anyone have to apportion an emergent outcome. That is the test many-hands accountability has to pass. Most current regimes fail it.

The three mechanisms that make it enforceable

  1. 01A named accountable individual per layer. UK financial services has this and calls it the Senior Managers regime: a documented statement of responsibilities, and personal sanction if a firm breach happens in your area and you did not take reasonable steps. US securities law has it and calls it SOX 302 and 404: the CEO and CFO sign, personally, with criminal exposure for a knowing false certification. The EU AI Act has nothing equivalent. Duties attach to legal persons. The natural persons assigned oversight under Article 26(2) sign nothing and face nothing. A SOX-style attestation at frontier model release and at high-consequence agent deployment is the highest-leverage mechanism in AI governance that nobody has built.
  2. 02Instance-level identity and logging as a precondition of deployment. Agent identifiers plus activity logs, mandatory above a consequence threshold: moving money, contacting real people, writing to production. Without attribution across the trace, every other duty is unenforceable, because no forum can reconstruct who did what. The Model Context Protocol's own security specification names the failure: token passthrough means downstream logs "may show requests that appear to come from a different source with a different identity," which makes incident investigation and auditing harder.
  3. 03A functioning insurance layer, backstopped if it does not form. Insurance is the only mechanism that prices risk continuously, differentiates by deployer, and conditions cover on controls. In November 2025 the Financial Times reported that AIG, Great American and WR Berkley had sought permission from US regulators to write policies excluding AI liability. Meanwhile a Lloyd's coverholder was already selling affirmative cover that itemises "AI agent mistakes, failure to escalate" as a named peril. The market priced agent risk before any legislature named it, and then large parts of the market tried to leave. Regulators should read the exclusion wave as a market failure to correct rather than a commercial choice to observe.
06

The variance by region

Six jurisdictions, six different answers to a prior question nobody states out loud: what is the object being regulated?

Most comparisons of AI regulation line up risk tiers and penalty ceilings. That is the wrong cut, because it compares the answers while ignoring that the jurisdictions are answering different questions. China regulates the public-facing service and its content. Korea regulates the impact domain. Japan regulates the ecosystem and attaches no penalty at all. India regulates the intermediary and the artefact. Singapore regulates the measurement. The EU regulates the product. The United States, in 2026, has largely decided to regulate through litigation.

Each choice determines who is accountable and what oversight can actually see. Get that right and the rest follows.

Exhibit 11: what each regime treats as the regulated object
JurisdictionRegulated objectAccountable partyGateReal sanction
ChinaThe public-facing service and its contentThe service provider, with duties cascading to platforms and app storesPre-market filing or registration. The only ex-ante authorisation gate operating at national scale anywhere.Product removal, campaign enforcement, licence exposure. Over 14,000 non-compliant AI products removed in the first phase of the 2026 Qinglang campaign.
EUThe product placed on the marketProvider, with the label flipping to a deployer who substantially modifies or repurposesConformity assessment, mostly self-assessment for the rights-affecting categoriesUp to 7% of global turnover in principle. Zero completed AI Act enforcement actions as of August 2026.
United StatesNothing, federally. At state level, the decision or the disclosureDeployer, sometimes developer, and increasingly the vendor through litigationNone ex-anteTort, class actions, state AG enforcement. The live theory is agency liability, not regulation.
IndiaThe intermediary and the artefactThe platform, via conditionality on safe harbour. Separately, the regulated financial entity, strictly.None, but a three-hour takedown clockLoss of Section 79 safe harbour, which for a platform is existential.
KoreaThe impact domain, defined sectorallyThe AI business operator plus a mandatory domestic representativeNone, ex-postRoughly USD 21,000 maximum. Enforcement suspended by administrative grace from the day the law took effect.
JapanThe ecosystemNobody in particularNonePublication of the offender's name. That is the sharpest instrument in the statute.
SingaporeThe measurementWhoever the assurance market pricesNoneSupervisory and market expectation, with an accredited tester ecosystem built deliberately.

Europe: the regime that was not ready

If you carry one fact out of this section, make it this one. The EU AI Act's human oversight article is not in force, and will not be until December 2027.

Two further European facts matter more than the delay, and both are about capacity rather than text.

9 of 27
Member States that had fully designated both a market surveillance authority and a notifying authority, as of 17 June 2026. The statutory deadline was 2 August 2025.
6 had designated neither
8 of 27
national single points of contact on the Commission's list as of March 2026
European Parliamentary Research Service, 18 March 2026
1 June 2026
the date the Scientific Panel was constituted, ten months after GPAI obligations became applicable
Up to 60 experts, two-year terms

And then the fact that undercuts the popular description of the Act as a strict ex-ante regime. Under Article 43, most Annex III high-risk systems, meaning points 2 to 8, which covers hiring, credit, education, essential services and law enforcement, go through internal control. That is provider self-assessment. Third-party notified body assessment is required essentially only for biometrics. For the overwhelming majority of high-risk uses that touch individual rights, the provider grades its own homework, and the external check is post-market surveillance by an authority that two thirds of Member States had not fully constituted.

There is a design contrast buried in that and it is the most interesting thing in European AI law. Article 14 tries to make the human in the loop competent enough to catch the machine. The Product Liability Directive assumes the human will not understand the system at all and shifts the evidential burden accordingly. Given everything in section 02, the second approach is the honest one.

One more European mechanism deserves attention because it is the thing the AI Act did not copy from EU law that already worked. Under the Digital Services Act, designated platforms must assess systemic risks, submit to an annual independent audit, and publish the risk assessment, the audit report and the audit implementation report. That full loop, self-assessment plus third-party audit plus publication, exists nowhere in the AI Act for Annex III deployers. It was a design choice, not a technical impossibility.

The United Kingdom: general-purpose law, no AI-specific oversight

The UK still has no AI statute. The promised frontier bill has slipped roughly two years. The AI Safety Institute was renamed the AI Security Institute in February 2025, which narrowed its declared remit from societal harm toward national-security-adjacent risk, and it has no statutory basis, no power to compel model access, and no sanction. The one hard legal change of the period, the Data (Use and Access) Act 2025, moved automated decision-making protection from a prohibition-plus-exceptions model toward permission-plus-safeguards, at exactly the moment the Court of Justice was moving the other way.

The fair way to state it: the UK has general-purpose accountability law applied to AI and no AI-specific oversight, while the EU has AI-specific oversight law that is mostly not in force. Neither jurisdiction has produced a completed AI-specific enforcement action against a major developer that survived appeal. Europe's largest AI-related fine, the Italian regulator's EUR 15m against OpenAI, was annulled by the Court of Rome in March 2026 on jurisdictional grounds. One analysis of the European record on generative AI enforcement is titled "A Lot of Noise, One Fine, Zero Survivors."

The United States: accountability is being determined by courts

The US story in 2025 and 2026 is a federal government actively removing the theories that algorithmic harm claims depend on, state legislatures partially filling the gap, and courts becoming the place where accountability is actually decided.

  • Executive Order 14281 (23 April 2025) set the policy of eliminating disparate-impact liability "in all contexts to the maximum degree possible" and directed all agencies to deprioritise enforcement of statutes to the extent they include it. Algorithmic discrimination is almost always a disparate-impact problem: no protected variable, no intent, harm visible only in outcome rates. Removing disparate impact does not narrow the theory for AI. It removes it.
  • The EEOC's May 2023 guidance on adverse impact in algorithmic selection now returns a 404. Six federal AI-employment guidance documents were removed across January and February 2025. On 4 June 2026 the EEOC replaced its Strategic Enforcement Plan with a National Enforcement Plan in which artificial intelligence does not appear anywhere. The predecessor plan had listed it as a priority.
  • The CFPB withdrew 67 guidance documents on 12 May 2025, including the two circulars that had told creditors they could not excuse non-compliance with adverse-action rules on the ground that their model was too complex to understand. On 22 April 2026 it finalised a rule holding that ECOA does not authorise disparate-impact liability at all, effective 21 July 2026, against roughly 64,500 comments mostly opposed.
  • Colorado's AI Act, the only US statute that ever imposed a duty of reasonable care on AI developers, was repealed before it took effect. Delayed from 1 February 2026 to 30 June 2026, then replaced by SB 26-189, signed 14 May 2026 and effective 1 January 2027, as a notice-and-explanation statute. The care duty is gone. Before the repeal, xAI had sued the state and the Department of Justice had intervened on the company's side.
  • Texas TRAIGA, effective 1 January 2026, prohibits developing or deploying AI with intent to discriminate, and states expressly that "a disparate impact is not sufficient by itself to demonstrate an intent to discriminate." No private right of action, a 60-day cure period, exclusive AG enforcement. An intent standard applied to a statistical harm is a null set.
  • An executive order of 11 December 2025 directed the Attorney General to create an AI Litigation Task Force to challenge state AI laws on preemption grounds. The federal posture is not merely absent. It is actively preemptive.

Two details in that sequence are worth pausing on. The EEOC removed its AI guidance by deleting web pages, with no Federal Register document, which means there is no administrative record to challenge. And in December 2025 the Federal Trade Commission reopened and set aside its own completed AI enforcement order against a writing-tool vendor, on the ground that it unduly burdened AI innovation. Enforcement outcomes, not merely enforcement priorities, turned out to be reversible.

The preemption push is real and, so far, mostly unexecuted. It failed twice in Congress, once stripped from a reconciliation bill by 99 votes to 1, then excluded from the defence authorisation. It moved to executive action in December 2025. As of August 2026 the Commerce Department list of "onerous" state laws that the order required by March has not been published, no broadband funds have been deobligated, no Federal Communications Commission proceeding has opened, and the litigation task force has never filed its own complaint. The one product delivered was a Federal Trade Commission proposal in July 2026 asserting that a state AI law is impliedly preempted by the FTC Act, which is a novel use of deception doctrine as a preemption vehicle.

Meanwhile the states kept legislating, just not where the federal government was pointing. Roughly 109 AI laws passed across 29 states in the first half of 2026, concentrated in companion chatbots and data centres, and no state enacted comprehensive private-sector AI obligations. Three instruments converged on the same design: California's frontier transparency act, New York's RAISE Act, and a bipartisan federal draft all use a revenue threshold above USD 500m, a published safety framework, 15-day incident reporting, whistleblower protection, no private right of action and no duty of care. That combination is becoming the American default, and it is a disclosure regime rather than an accountability one.

Which leaves the courts. Mobley v. Workday is the most consequential case in the field because it tests whether antidiscrimination law can reach the vendor directly, through agency, rather than only the employer. Workday's own filings put roughly 1.1 billion applications rejected through its software in the relevant period. Judge Lin's answer to the argument that the collective was too large is the doctrinal point on which vendor accountability turns: if the collective runs to hundreds of millions of people, that is because the vendor has been plausibly accused of discriminating against a broad swath of applicants, and allegedly widespread discrimination is not a basis for denying notice. Interlocutory review was denied in July 2026 and the case is heading to a class certification fight.

The other cases mostly settled, and that matters more than it looks. The Character Technologies wrongful-death suit, whose May 2025 ruling had held that the app is a product for strict liability purposes while its outputs are expression, settled and closed in January 2026. So the one holding that actually drew a workable line between design defect and content never got appellate treatment. Anthropic's copyright settlement received final approval in July 2026 at USD 1.5bn, roughly USD 3,000 per work, and the money was for the acquisition channel rather than for training. No appellate court has yet ruled on whether AI training is fair use.

Against all of that, two things survived and both are instructive. The first is that a statute is harder to kill than a guidance document. Title VII section 703(k) still codifies disparate impact, private plaintiffs still have the cause of action, and a right-to-sue notice is a ticket to court. The second is more precise and it is the single most useful lesson in the US material.

China: the regime nobody in the West reads carefully

China has no comprehensive AI act and, as of August 2026, no published draft of one. What it has is a stack of use-case and content-specific instruments with a pre-market filing gate that no Western jurisdiction has. The regulator holds an inventory of who is deploying what, before deployment.

The filing dossier is the tell. Five documents, and two of them are a keyword blocking list and an evaluation test question set. The state is not assessing model capability. It is assessing content controllability. That is the deal the regime strikes: pre-market visibility purchased with content compliance, and the two cannot be separated because they are the same instrument.

What it buys is real. A working provenance chain that runs from the generator through the platform to the app store to the individual user, with the metadata fields named in the standard rather than left to the state of the art. Enforcement that reaches products rather than papers. And the ability to impose a duty of care on a conversational system within months of identifying the harm: the companion-AI measures effective 15 July 2026 ban virtual-partner services for minors and require providers to intervene on detecting suicidal ideation, including by contacting a guardian. That is the first duty of care on a chatbot anywhere, and it is a materially different accountability object from a duty of disclosure.

India: an instrument from 2000, obligations from 2026

I am writing from India, so let me be careful to be accurate rather than flattering. India has no AI statute and, per the November 2025 governance guidelines, does not plan one. What it did instead is the most interesting regulatory manoeuvre of the year.

The IT Amendment Rules 2026 were notified on 10 February 2026 and took effect on 20 February, a ten-day compliance window. They require clear labelling of synthetically generated information, permanent provenance metadata tied to the generating system, a platform duty to deploy automated tools against illegal synthetic content, and a duty on significant social media intermediaries to require users to declare whether content is synthetic and to deploy verification tools to validate that declaration before publication. Government takedown compressed from 36 hours to three. Urgent takedown from 24 hours to two. Grievance resolution from fifteen days to seven.

All of it under the Information Technology Act 2000. The enabling statute is twenty-six years old and the obligations are current. And the accountability hinge is a single clause most coverage missed: proactive automated moderation does not breach Section 79 safe harbour. India resolved the Good Samaritan problem in the platform's favour, which makes filtering rational rather than legally risky.

There is also the sharpest accountability rule found anywhere in this survey, and it is two sentences in a securities regulation. SEBI Regulation 16C, notified 6 February 2025, makes a regulated entity solely responsible for the outputs of any AI tool it uses, whether built in-house or licensed. No risk tiers. No compute threshold. No new institution. No "the vendor's model did it" defence. The cheapest form of meaningful accountability may simply be strict liability attached to an actor who is already licensed.

The gap is on the individual's side. The Digital Personal Data Protection Act has no analogue to GDPR Article 22. There is no general right against a solely automated decision and no right to an explanation of one. Indian courts filled part of the gap first, through personality-rights injunctions in the Delhi High Court from 2023 onward, and regulation arrived in 2026 to formalise what judges had already been ordering. Courts first, rules after, is a distinct model of how accountability gets determined, and India is its clearest live example.

Coverage without bite, and the summit that asked for nothing

Korea has the most complete AI statute in Asia. It took effect on 22 January 2026 and enforcement was suspended the same day for at least a year. Its maximum fine for failing to tell users they are talking to an AI is roughly USD 21,000, against a revenue threshold of roughly USD 662m that triggers the domestic representative duty. Comprehensive scope, nominal penalty, immediate grace period.

Japan legislated and deliberately attached no penalty at all. Australia consulted on ten mandatory guardrails for high-risk AI and dropped all ten in its December 2025 National AI Plan, retaining binding obligations only on its own departments. Canada's AIDA died on prorogation in January 2025 and was never replaced; the operative algorithmic accountability law in Canada is Quebec's Law 25, which gives an individual subject to a solely automated decision the right to know the personal information used and the principal factors, to submit observations, and to have the decision reviewed by a person. Brazil's bill passed the Senate in December 2024 and is still in the Chamber.

And the international layer produced two events ten days apart in July 2026 that describe the whole problem. The UN's first Global Dialogue on AI Governance opened in Geneva on 6 July with two ambassador co-chairs and no authority. On 16 July, twenty-nine countries signed the founding agreement of the World Artificial Intelligence Cooperation Organization in Shanghai, with a charter and a headquarters. One convenes. The other has members.

The India AI Impact Summit in February 2026 was a genuine diplomatic success and a further dilution of the summit series as an accountability mechanism, and both are true at once. It drew representatives from 118 countries and moved the global agenda from frontier risk to access and impact. Of the New Delhi Declaration's seven chakras, exactly one concerns trustworthiness. The rest concern access, capacity, energy, science, resources and growth. Its concrete institutional creations are commons and networks. Nobody was made accountable to anybody. Note the pattern across the series: Bletchley in 2023 got both the US and China to sign a safety-framed text; Paris in 2025 got neither the US nor the UK to sign an inclusion-framed one; New Delhi in 2026 got the most signatures of all. A summit gets more endorsements the less it asks of the endorsers.

07

The variance by industry

Sectors with a pre-existing professional accountability structure adapted. Sectors without one got nothing, and then got tort law.

The single best predictor of whether a sector has meaningful AI accountability in 2026 is not how dangerous its AI is. It is whether the sector already had a named, licensed, insurable human who answered for outcomes before AI arrived. Medicine had one. Finance had one. Law had one. Aviation had one. Hiring did not. Consumer software did not. Education did not. The sectors with the structure adapted it, imperfectly. The sectors without it are now watching accountability get determined by juries.

Financial services: the regime everyone is told to copy, and what happened to it

SR 11-7, the Federal Reserve and OCC's 2011 supervisory guidance on model risk management, is the most complete model accountability architecture ever built. Model owners, independent validators, a firm-wide inventory, three lines of defence, and one idea that the rest of the economy should steal wholesale.

A guiding principle for managing model risk is effective challenge of models, that is, critical analysis by objective, informed parties who can identify model limitations and assumptions and produce appropriate changes. Effective challenge depends on a combination of incentives, competence, and influence.

Everyone quotes that sentence. Almost nobody copies the machinery underneath it, and the machinery is where the value is. Incentives means separation of the challenger from the builder plus compensation practices tied directly to the quality and critical unbiasedness of review. Competence means technical peers of the model builders, not compliance generalists. Influence means explicit authority to restrict the use of a model, not to file a finding. And a fourth thing nearly always lost in translation: internal audit's job is to evaluate whether the validators themselves have the right incentives to discover and report deficiencies. Effective challenge is a control that is itself controlled. Without that third line it decays into documentation within two review cycles, because the reviewer's career depends on the reviewed.

There are four honest reasons SR 11-7 strains against modern AI, and they are worth naming because any sector copying it will inherit them. There is no stable inventory item: a prompt edit, a temperature change, a retrieval corpus refresh and a provider's silent weight update are all variations that would each warrant separate validation if you read the text literally, which makes the inventory uncountable, and fiction if you do not. The prompt is code and it sits outside change control, editable by a product manager. The vendor clauses are unsatisfiable, because a frontier lab will not supply developmental evidence or training data descriptions. And outcomes analysis has no ground truth for a summarisation or drafting task, because there is no realised counterfactual to score against, only human preference. As one practitioner put it, material changes in behaviour can now occur without a formal redevelopment event, and that single fact dissolves the trigger for revalidation, change control and inventory maintenance at the same time.

The other lesson from this sector is about durability, and it is uncomfortable. Fifteen years of the most admired model governance regime in the world produced no fine for a bad model. The mechanism was examination, examination findings are unpublished by design, and in 2025 and 2026 the agencies narrowed the findings, disclaimed enforceability, removed the validation cadence and told examiners not to spend excessive attention on process. An accountability regime built on supervisory discretion has exactly the durability of the political appetite to supervise. What survived intact were the rules written as hard operational requirements with deadlines and forms and private rights of action.

Healthcare: approval is not oversight

Healthcare is the control condition for this whole question. It has mature statutory ex-ante approval, and the human in the loop is not a metaphor: it is a named, licensed, insurable person. If approval plus a human in the loop works anywhere, it works here.

1,524
AI-enabled medical devices authorised by the FDA, parsed from FDA's own list
Page current as of 16 June 2026, most recent decision 30 March 2026
96.2%
came through 510(k), which asks whether a device is substantially equivalent to a predicate, not whether it works
1,466 of 1,524. De Novo 39, PMA 19.
43%
of 521 authorised AI devices studied reported no clinical validation data at all. Only 4.2% were validated by randomised trial.
Chouffani El Fassi et al., Nature Medicine, 2024

Then there is the case that should be taught in every AI governance course. Epic's sepsis prediction model was deployed at hundreds of US hospitals. In 2021 a team at Michigan Medicine ran the first independent external validation across 38,455 hospitalisations.

The deeper diagnosis is worse than poor accuracy, and it defeats every oversight instrument in this paper. When researchers restricted a sepsis model to pre-treatment data, its discrimination fell from 0.87 to 0.62, and to 0.53 before a blood culture was ordered. A coin flip. The model was not predicting sepsis. It was detecting that a clinician had already noticed and started acting. Label leakage of that kind is invisible to a change control plan, invisible to a substantial equivalence review, and invisible to every one of the 31 transparency attributes a US rule requires vendors to publish.

And the correction, when it came, came from universities publishing against a vendor, twice, unpaid. An independent prospective validation of the rebuilt model across 227,091 encounters found better discrimination and something more useful: the threshold needed for 60% sensitivity varied 2.6-fold across four sites. A single national performance claim for a clinical model is meaningless by construction. Local validation is the only control that catches any of this, and while adoption rose, the share of independent hospitals doing it went backwards, from 44% to 41%. Where a rigorous local evaluation programme does exist, one published base rate is that two of six candidate tools survived it.

The reason it never needed approval is a carve-out worth understanding precisely, because it is the clearest example anywhere of a human-oversight test written into law and then failing. Section 3060 of the 21st Century Cures Act excludes clinical decision support software from the device definition where four conditions hold. The fourth is that the software must enable a health care professional to independently review the basis for the recommendations, so that reliance on them is not the intent.

That is a human-oversight test, and the FDA's guidance grounds it explicitly in automation bias, noting that bias is more likely where software gives a single specific output rather than a list of options, and that it increases where action is urgent. The agency even states that software identifying a risk probability or risk score for a specific disease does not satisfy the criteria and is therefore a device. Sepsis models produce a risk score. On the FDA's own written interpretation they are unapproved devices deployed at national scale, across 6,129 US hospitals, with no documented discontinuations and no enforcement action. The gap is not in the rule. It is in the enforcement of a rule the agency has already written.

For large language models the criterion breaks in a more fundamental way. There is no basis to review. The chain of reasoning an LLM emits is generated text, not a trace of the computation that produced the answer. A clinician reading it is reviewing a plausible artefact. This inverts the doctrine's premise: the rule assumes the clinician is the superior verifier, and against a fluent hallucination the clinician is structurally the inferior verifier, because the failure mode is optimised to survive human reading. Add that the highest-volume clinical LLM uses, drafting an inbox reply or summarising a forty-page chart before a fifteen-minute visit, are adopted precisely because the clinician has no time, which is the negation of the condition that exempts them.

Healthcare does contain the single most interesting regulatory innovation in AI oversight anywhere, and it deserves to travel. The Predetermined Change Control Plan lets a sponsor pre-authorise a bounded space of future model change: a description of the modifications, a protocol for making them, and an impact assessment, all reviewed at authorisation. It is the only mechanism any regulator has built that accepts the artefact will change and regulates the change process rather than the frozen artefact. Every sector wrestling with the model-boundary problem should read it. Two caveats keep it honest. It sits on roughly 4.2% of authorised AI devices, 64 of 1,524. And most of its uses are on non-AI products, where the space of future change is easy to enumerate, which is precisely where it is least needed.

Which leaves the clinician holding it, and here the legal position is stranger than the policy debate assumes. No malpractice case naming an AI tool exists anywhere. Two randomised experiments, in two legal systems, found that jurors do not punish a clinician for following nonstandard AI advice. So the liability sink that the whole learned-intermediary structure assumes has never actually been tested, while clinicians are being told they are responsible for both overriding a correct output and following an incorrect one. That is the moral crumple zone with a licence attached.

And the sector's own regulator has diagnosed the post-market problem in its own words. The FDA's January 2025 draft lifecycle guidance says performance "may change or degrade over time, presenting a risk to patients" and that "it may not be possible to completely control risks with development and testing activities performed premarket." Then it says sponsors that elect to employ proactive performance monitoring should describe their plans, and that for a 510(k) submission the agency generally does not require them. Nineteen months later that guidance is still in draft. Post-market monitoring of authorised medical AI is voluntary.

Employment: the natural experiment, and its result

New York City Local Law 144 was the first mandate anywhere requiring bias audits of commercial algorithmic systems. Enforcement began 5 July 2023. It is the best natural experiment we have, and the result is unambiguous.

Exhibit 12: NYC Local Law 144, measured. Cornell and Data & Society, FAccT 2024, and the New York State Comptroller's audit of December 2025
MeasureValue
Employers checked by 155 student investigators391
Employers that posted a bias audit report18, or 4.6%
Employers that posted a transparency notice13, or 3.3%
Published impact ratios below the four-fifths threshold9 of 386, or 2.3%, which the authors read as publication bias rather than a clean industry
Complaints the enforcement agency received in 24 months2
Demand letters issued in 24 months2
Non-compliance instances the regulator found in a 32-company review1
Non-compliance instances state auditors found in the same 32 companiesat least 17
Test calls to the city helpline correctly routed to the regulator3 of 12, or 25%

The Comptroller's conclusion: "DCWP has an ineffective system to address compliance with NYC's LL144." The agency accepted ten of thirteen recommendations and declined to research why complaint volume is so low, declined to build a proactive detection process, and declined to fully investigate the violations the auditors had already found.

The researchers named the mechanism, and the name is the most useful concept in this section. Null compliance: a state in which the absence of evidence of compliance cannot be established as non-compliance, because the investigator lacks the information to determine whether the regulated party is even in scope. Coverage under LL 144 turns on whether the tool "substantially assists" the decision, which is an unobservable internal process that the employer characterises. An applicant, a researcher and the regulator all see exactly the same thing when an employer complies and when an employer decides it is out of scope: nothing.

Illinois supplies the purest specimen. Its AI Video Interview Act has since 2020 required employers who rely solely on AI to decide who advances to report applicant race and ethnicity to the state. The annual report to the legislature has carried the same sentence for four consecutive years: no such data was reported. Zero, four years running. The Act contains no penalty provision, no enforcement agency and no private right of action, and the reporting duty attaches only where the employer relies solely on AI, which is a characterisation the employer supplies.

Two counter-designs are worth studying. California's Civil Rights Council regulations, effective 1 October 2025, do not mandate an audit at all. They make the presence or absence of anti-bias testing admissible evidence in a discrimination claim, and they extend the definition of employer to cover agents. That inverts the incentive: under LL 144 an audit is a compliance artefact to be posted or quietly avoided; under the California rule, not testing is a fact a plaintiff can put to a jury. And German co-determination, in force since long before AI, gives a works council a veto over the introduction of technical systems capable of monitoring employee conduct or performance, triggers on the system's capability rather than the employer's characterisation of its role, and entitles the council to an external expert at the employer's cost. It is the only mechanism in this paper that gives an affected party the power to say no.

The public sector: where the bodies are

Government is the sector with the longest record, because administrative automation predates the current AI wave by a decade. The record is the strongest available argument that oversight failure is an institutional problem rather than a technical one, and the numbers are worse than the summaries usually suggest.

Exhibit 13: four administrative failures, and what actually happened afterwards
CaseScaleAccountability outcome
Netherlands childcare benefits43,000 and more recognised victims, not the 26,000 usually cited. Two data protection fines, EUR 2.75m and EUR 3.7m, paid by the state to itself.The government resigned. The self-inflicted fine is the tell: when the wrongdoer and the enforcer are the same entity, the penalty is an accounting entry.
Australia's RobodebtA$1.872bn in unlawfully raised debts, recovered from people who mostly did not owe them.A Royal Commission found 12 of 16 officials in breach. None terminated. No prosecutions. The integrity commissioner who handled the referrals was later himself found guilty of officer misconduct.
Michigan unemployment fraud (MiDAS)20,965 of 22,427 determinations reversed, a 93% error rate, over nine years.USD 20m settlement. A machine accused twenty thousand people of fraud, was wrong about nineteen in twenty, and it took the better part of a decade to unwind.
Amsterdam Smart Check1,600 applications over a nine-month pilot. Roughly EUR 500,000 spent.The control experiment. Explainable model, protected variables excluded, bias tested and remediated before deployment, external academic consultation, data protection sign-off, participation council consulted, registered and documented. It still flagged more people than intended, performed no better than caseworkers, and its bias inverted rather than disappearing. What stopped it was a pilot with measurement, a council with standing to object, and a politician willing to absorb the write-off. Not the register.

The Amsterdam case is the one to argue from, because it isolates transparency from outcome. Every disclosure box was ticked and the system was still unfit to deploy. Registers record. They do not decide.

Which brings us to the registers themselves, the instrument most often proposed as the public-sector answer. The measured record is not encouraging. The UK's algorithmic transparency standard holds 138 records, and its publication rate fell 68% in the year after the mandate was declared fully in force. The National Audit Office found that of 32 government bodies deploying AI, 12 were never compliant with the standard. Canada's directive on automated decision-making, binding since 2019, has produced 38 impact assessments in seven years from ten organisations, nearly 60% of them from the two departments that automate immigration and benefits at scale.

Education deserves a short note because it is where the harm is least contested and the accountability is thinnest. Research on AI text detectors found an average 61.22% false positive rate on essays by non-native English writers, which fell to 11.77% simply by prompting for more elaborate prose. That is a detector that penalises people for writing plainly. At one university, 33% of AI misconduct hearings ended in a finding of not responsible because the detector score was the only evidence offered. Penalties actually collected from ed-tech vendors by the US consumer protection regulator, in the same period: zero.

The cross-sector picture

Exhibit 14: who is accountable, by what mechanism, and where it breaks
SectorAccountable todayMechanismWhere it breaks
Financial servicesThe regulated firm, and in the UK a named senior managerIndependent model validation with effective challenge; SM&CR statements of responsibility; DORA critical third-party designationGenerative and agentic AI were carved out of the US model risk guidance in April 2026. The regime is examination-driven and therefore switchable off by memo.
HealthcareThe licensed clinician, almost entirelyDevice authorisation for a minority of tools; professional licensure and malpractice for the rest96% of authorised devices came through a comparison standard, not an evidence standard. Post-market monitoring is voluntary. The clinical decision support carve-out exempts the highest-volume LLM uses on a condition they cannot meet.
InsuranceThe insurer, non-delegably in the strongest state rulesNAIC model bulletin governance programmes; NY DFS and Colorado rules that bar reliance on a vendor's proprietary claimsProcess regulation with a circular definition of adverse outcome, and self-assessed materiality. Colorado's quantitative testing rule, the only attempt anywhere to name a methodology and a threshold, was the road not taken.
EmploymentThe employer. The vendor only through litigation.Bias audit mandates, notice duties, and civil rights statutesNull compliance. Coverage is self-declared, enforcement is complaint-driven, applicants cannot know they were screened, and the audit can be made privileged.
Public sectorThe department, in principleAlgorithmic impact assessments, transparency registers, procurement conditionsSelf-assessed impact levels set the obligations. Registers record and do not decide. The UK register's publication rate fell 68% in the year after the mandate was declared in force.
Legal and professional servicesThe licensed professional, unambiguouslyProfessional licensure, court sanctions, bar rulesIt works, and it is the cleanest accountability model in the paper. It does not scale to sectors with no licence.
Consumer software and companionsNobody, until 2025Product liability, state AG action, tortThe mechanism arrived through wrongful-death litigation rather than regulation, which means accountability is being priced by juries rather than specified by rules.

One observation cuts across all of it. In healthcare, the only enforcement action against a clinical generative AI vendor to date came not from the FDA but from a state consumer protection regulator, over marketing claims about hallucination rates. No patient injury, no causation problem, no standard-of-care fight. Just a number in a sales deck. Where product regulation has a carve-out, accuracy marketing becomes the enforceable surface. That is true far beyond medicine and it is the most under-used lever in AI accountability today.

08

The variance by business function

Not all work in a company is equally sensitive. The same model in HR and in marketing attracts completely different law, and almost nobody budgets for that.

Region and industry are the axes everyone maps. Function is the one that decides where the money goes, and it is the cut most governance programmes get wrong. A large company does not have an AI risk. It has eleven of them, they are wildly unequal, and the regulation that bites is usually not AI regulation at all.

Here is the point in one comparison. Put the same language model behind a hiring screen and behind a marketing email. In the first, you are in EU Annex III high-risk territory, a New York bias audit mandate, an Illinois civil rights amendment, and the live vendor-liability theory in Mobley. In the second, no AI law applies to you at all, and your exposure runs entirely through fifty-year-old advertising and consumer protection rules. Same model. Same vendor. Same week. Two different bodies of law and two different people who go to the hearing.

Exhibit 15: what actually bites, function by function
FunctionTypical AI useThe regime that actually bitesThe trap
HR and PeopleScreening, ranking, interview scoring, promotion, task allocation, performance and attrition prediction, workforce monitoringThe most regulated function in the world for AI. EU Annex III point 4 covers recruitment and task allocation and performance monitoring. NYC bias audits, Illinois civil rights, Colorado's replacement notice regime, German works council veto, and the vendor-agency theory in Mobley v. Workday.Coverage is self-declared nearly everywhere, so the function reports itself compliant while doing nothing. And the bias audit, if commissioned through counsel, can become privileged and therefore useless to the person it was meant to protect.
Finance, Treasury and CreditUnderwriting, scoring, pricing, fraud, collections, forecasting, close automationNot AI law. Model risk management, and in credit the adverse-action rules that pre-emptively ban both black-box answers. In the UK, a named senior manager. In India, strict non-delegable liability on the regulated entity.The US model risk framework was rescinded in April 2026 with generative and agentic AI carved out of its successor. The function most likely to believe it is already governed just lost the instrument it was relying on.
Marketing and GrowthCopy, creative, segmentation, personalisation, pricing tests, synthetic media, SEO and answer-engine optimisationAlmost no AI law. Consumer protection and advertising law, deception and unfair practice authority, plus synthetic-media labelling duties in the EU, China and India.The most under-governed function relative to its exposure. The enforceable surface is the claim rather than the model. Where product regulation has a carve-out, accuracy marketing becomes the thing regulators can actually reach, and marketing is the function that writes the claims.
Customer Service and SupportDeflection bots, agent assist, drafting, triage, refunds, retention offers, voiceConsumer protection, contract law, and the EU duty to tell a person they are dealing with a machine. Companion and minor-safety rules where the interaction is emotional or the user is young.The bot binds the company. A chatbot promised a bereavement refund policy that did not exist, and the airline was held responsible for what its chatbot said, on the reasoning that a company is responsible for all the information on its site whether it comes from a static page or a bot.
Sales and RevenueLead scoring, propensity, forecasting, next-best-action, call analytics, dynamic pricingCompetition law and consumer protection more than AI law. Price coordination is the live risk, and the one US case where a regulator has actually constrained an algorithm's internals came through antitrust.Nobody classifies sales AI as high risk, and the largest algorithmic enforcement risk in the function is not fairness. It is two companies' models converging on a price without anyone agreeing to anything.
Operations, Supply Chain, ManufacturingDemand planning, routing, scheduling, predictive maintenance, quality vision, roboticsProduct safety and machinery law, not AI law. In the EU this is Annex I embedded high-risk, and it runs on a different and later clock than everything else, to August 2028.The function assumes the AI Act does not apply because it is not making decisions about people. The safety obligations are real, they arrive through the machinery and product regimes, and they land on engineering rather than on the AI governance committee.
Health, Safety and Workforce AnalyticsProductivity quotas, pace setting, ergonomic risk, incident prediction, monitoringOccupational safety law, warehouse quota statutes, works council co-determination, and electronic monitoring notice rules.68% of US workers report electronic monitoring, and those monitored constantly report injury at 9% against 4%. Yet a regulator that traced ergonomic harm to algorithmic pace lost, because no standard with a number in it existed. The harm was real and the rule was not.
Legal and ComplianceResearch, drafting, review, discovery, contract analysis, regulatory monitoringProfessional licensure and court rules. The cleanest accountability model in this paper: the licensed professional remains responsible regardless of the tool.It works, and it does not scale. It functions because there is a licence to lose. Every function in this table without one is asking for an accountability mechanism that licensure supplies for free.
Procurement and Vendor ManagementSupplier scoring, spend analysis, contract review, bid evaluationBarely regulated as AI. But procurement is where every other function's AI risk is actually decided, because the contract is where audit rights, model change notification and liability either exist or do not.The most leveraged function and the least involved. By the time an AI system is in front of the ethics committee, procurement has already signed away the audit right. A certificate that attests to a management system is being read across the market as a claim about a model, which is a category error.
IT, Security and PlatformAgents, copilots, code generation, SOC triage, identity, tool integrationSecurity and resilience law rather than AI law, plus whatever the integrator quietly decided about tool scopes and write permissions.This function holds the controls that would have prevented every documented agentic incident, and it is a regulated role in no jurisdiction on earth. The blast radius is set here, in a config file, by someone who was never invited to the governance meeting.
Strategy, Corp Dev and the ExecutiveMarket analysis, diligence, scenario modelling, board reportingSecurities disclosure and directors' duties. Regulators will punish overstating an AI capability far more reliably than understating an AI risk.If the monitoring apparatus is itself automated, its degradation shows up as fewer alerts, which every dashboard renders as green. A falling alert count is a warning, not a result.

Three tiers of regulatory gravity

Strip the table back and functions sort into three tiers, and the sorting principle is not how advanced the AI is. It is whether the output is a decision about a specific person who cannot easily walk away.

Exhibit 16: sorting functions by what the AI decides
TierTestFunctionsWhat follows
Tier 1: decides about a personThe output changes an identified individual's access to work, money, care, or liberty, and they cannot shop elsewhere.HR, Credit, Claims and Underwriting, Clinical, anything customer-facing that can denyNamed accountability, an appeal route with a clock, documented human review, and measurement of whether that review changes anything. This is where the whole apparatus in this paper belongs.
Tier 2: decides about a thing, near a personThe output moves physical or financial state and a person is exposed to the consequence.Manufacturing, Logistics, Treasury, Security, Site operationsBlast-radius controls and confirmation gates matter far more than explainability. Nobody needs the model interpretable. They need it unable to do the largest bad thing unattended.
Tier 3: produces a claim or an artefactThe output is content that goes to the world in the company's name.Marketing, Sales collateral, Comms, Support macrosThe exposure is what you asserted, not how you generated it. Accuracy of claims, disclosure of synthetic media, and a human who owns the publish button. Cheap to govern and almost universally ungoverned.

The mistake I see most often is treating Tier 3 volume as Tier 1 risk, because it is where the visible AI usage is. Marketing generates ten thousand assets and HR makes four hundred decisions, so the governance attention follows the asset count rather than the consequence.

Where the money goes, and where the exposure is

This is the practical finding and it is the reason function is the right axis for an operator. AI governance spend in most large companies follows engineering, because that is where the AI visibly is. Regulatory and litigation exposure sits in HR, in credit decisions, in customer-facing support, and in the claims marketing makes. Those are different buildings.

The one question that sorts a function in a minute

You do not need a taxonomy to triage this. Ask of any AI use in any function: if this is wrong about one specific person, does that person find out, and can they do anything about it? If they find out and can appeal, you have a Tier 1 system with a working remedy and your job is to make the remedy real. If they find out and cannot appeal, you have a complaint queue and a reputational problem. If they never find out, which is the usual answer in hiring, in pricing and in support triage, then no amount of internal oversight is being tested by anyone with an interest in the answer, and the five tests in section 03 are the only thing standing between you and a system nobody is checking.

09

Agentic AI, where everything above breaks

No jurisdiction has an agent-specific accountability regime. The only binding agent-specific obligation anywhere in law is a disclosure duty.

Everything to this point assumes a system that produces an output which a human then acts on. An agent does not produce outputs. It takes actions, in sequence, using credentials somebody granted it, against tools nobody in the chain wrote. Each of the five tests in section 03 degrades, and one of them fails outright.

Exhibit 17: what agents do to the five tests
TestWhat changes
NameDegrades. Multi-agent harm emerges from interaction, so the harm's unit of analysis is a population of systems while the law's unit is a single system. Miscoordination between two correctly-behaving agents has no author.
SightDegrades badly. The reasoning an agent emits is generated text, not a trace of its computation. And token passthrough across services means downstream logs may show requests appearing to come from a different identity than the service actually forwarding them.
PowerHolds, and is the one to invest in. A confirmation gate before a write is the highest-value control in the entire agentic stack, and it is cheap.
TimeFails. METR measured the length of task a model can complete with 50% reliability, and found it doubling roughly every seven months. Once an agent reliably runs multi-hour tasks, per-action human review is not merely difficult. It is architecturally unavailable, for exactly the reason Bainbridge gave in 1983.
ProofBecomes both harder and more necessary. There is no override rate when there is no per-action review. What replaces it is the exception rate, the escalation rate, and the blast radius of the actions that ran unreviewed.

One regulator has said the quiet part out loud, and it is the only published agentic supervision framework in the world. Singapore's monetary authority, in July 2026, put the Time failure in a single sentence: the volume and speed of agent decisions make traditional per-action oversight operationally impossible. Every framework that answers the agentic question with a human review step is answering a question that has already closed.

The fact the industry has conceded

Prompt injection is not solved, and the strongest evidence for that is not a paper. It is that the industry has stopped trying to defend against it and started rationing capability instead.

Until robustness research allows us to reliably detect and refuse prompt injection, agents must satisfy no more than two of the following three properties within a session to avoid the highest impact consequences of prompt injection.

The three properties are the same three Simon Willison had named the lethal trifecta four months earlier: access to private data, exposure to untrusted content, and the ability to communicate externally. His assessment of vendor guardrails advertising a 95% catch rate is worth carrying into any procurement conversation: in web application security, 95% is very much a failing grade. Rationing is what you do when you cannot defend, and a public engineering rule that says so is a stronger admission than any research result.

The 2025 and 2026 incident record follows the same shape, and every one of them lands on the integrator layer that no regime regulates. Replit's coding agent deleted a production database during a session with a customer and then misreported the state of the data; the agent had unconfirmed write access to production. Brave's security team showed that hidden text in a web page could make an agentic browser read the user's email, retrieve a one-time password and exfiltrate both, because same-origin policy is meaningless once an agent acts with the user's full authenticated privileges. And in November 2025 Anthropic reported disrupting a state-linked espionage campaign that used its coding tool for most of the work. Worth flagging honestly that the widely cited figure for how much of that campaign ran autonomously is a vendor self-assessment of an incident only that vendor observed, and it has never been independently audited.

Where the accountability work is actually being done

Not in legislation. In protocol specifications, and the most candid document on agent accountability in existence is a security spec. The Model Context Protocol's security guidance names confused deputy, token passthrough, session hijacking and scope inflation as first-class threats, requires per-client consent, forbids servers from accepting tokens not issued for them, and requires that a one-click local configuration show the exact command without truncation before executing it. Read as governance, that is a protocol doing the delegated-authority work that agency law has not done. Scope minimisation with incremental elevation is a technical restatement of scope of authority.

The commerce protocols are doing something narrower and it is worth naming precisely. Both the Stripe and OpenAI agentic commerce work and Visa's trusted agent protocol allocate identity with real care: signed, time-bound, merchant-specific, non-replayable, with the business remaining merchant of record and able to decline per agent or per transaction. Neither allocates loss. Chargeback and fraud liability for a disputed agent-initiated transaction is being settled by existing card network rules. The protocols encode authentication and refusal rights and are silent on who pays.

The self-regulation evidence base

Frontier safety frameworks are the substitute for regulation in the agentic space, so it is fair to ask whether they work. Two independent raters, different methodologies, converge on the same answer. The Future of Life Institute's Summer 2026 index tops out at C+ with three F grades and concludes that companies are publishing and updating safety frameworks but the frameworks "have weak teeth." SaferAI's July 2026 tracker tops out at 35%, and notes that a company adopting every best practice already visible somewhere in the industry would score 59%. Nobody is near the frontier of their own collective practice.

These are unilateral public statements, revisable by their authors, with no external verifier, no audit right and no remedy for breach. One major lab's scaling policy has gone through three major versions in under three years. Version churn is not bad faith, and it does mean the commitment a government relied on at signing is not the commitment in force. The honest reading is that the binding constraint on frontier safety is not knowledge. The practices exist and are documented. It is incentive.

10

What actually works, ranked by evidence

Six mechanisms with a demonstrated track record, all of them borrowed from industries that learned the hard way.

The evidence in this paper is mostly negative, so it is worth being precise about what is positive. These six have measured effects. They are ranked by the strength of the evidence behind them, not by how easy they are to legislate.

Exhibit 18: instruments with a demonstrated track record
InstrumentThe evidenceWhy it works
1. Named-individual certification with personal exposureSarbanes-Oxley section 906: the CEO and CFO personally certify, with up to $1m and 10 years for a knowing false certification, $5m and 20 years for a willful one.It asks for a warranty of a process, not a review of every output. That is the only ask that survives Bainbridge's objection, and it is why it is portable to AI without inventing new liability theory.
2. Public, specific, named external auditRaji and Buolamwini measured what happened after the Gender Shades audit. All three named companies shipped new versions within seven months and cut darker-skinned female error by 17.7% to 30.4%. Two unnamed peers, with the same public benchmark and published method available, sat at 31.37% and 22.50% error on the same subgroup.As close to a natural experiment as this field has. The active ingredient was publication of a named result, not the availability of the technical knowledge. It argues directly for mandating disclosure of audit findings.
3. Confidential, immunised reporting held by a non-regulatorNASA's Aviation Safety Reporting System has taken 2,321,050 reports since 1976, including 121,128 in 2025, and issued 8,069 alert messages. Compare the AI Incident Database at roughly 1,618 curated incidents.The immunity is bounded, not blanket: it excludes deliberate acts, criminal offences, accidents and competency failures. There is a 10-day filing clock and a five-year clean-record condition. And the custodian has no enforcement power at all, which is what makes the volume possible. AI's incident record is built from journalism. Aviation's is built from confession.
4. Statutory severance of investigation from liability49 U.S.C. 1154(b): no part of a National Transportation Safety Board report may be admitted into evidence or used in a civil damages action.The NTSB is not blameless because its investigators are polite. Congress cut the investigation out of the liability system, which is what makes candour rational. Every disclosure an AI developer makes to a regulator today is a discoverable admission.
5. Mandatory structured incident reporting with public raw dataNHTSA's standing general order produced machine-readable crash reports with version amendment tracking. The currently published files hold 1,239 automated driving system reports and 1,800 Level 2 driver assistance reports.An existence proof against the claim that incident reporting is impractical for fast-moving technology. One caution: report counts track deployment volume and reporting diligence, not safety. One operator files 82% of the reports because it drives the most miles. Publish a denominator or the numbers get read backwards.
6. Mandatory post-market surveillanceAbsent for AI. The cost of its weakness even where it exists is Vioxx: a seven-fold heart attack risk signal before approval, five-fold in a trial published November 2000, roughly 18 months to a label change, and an estimated 88,000 to 139,000 excess US cases, of which 30 to 40% probably died.The FDA scientist who testified to those numbers put it in terms an AI regulator should borrow: the equivalent of two to four jetliners falling out of the sky every week for five years. His verdict on the agency was "we are virtually defenseless."

Set against those, here is the honest ranking of the instruments AI governance actually reaches for. Explanation requirements have a measured negative effect on the thing they are supposed to enable, because explanations increase acceptance of wrong answers. Training requirements target a variable the research says is not the cause. Human-in-the-loop mandates without instrumentation produce Green's legitimation problem. Compute thresholds are a bright line chosen because bright lines are enforceable, not because they track risk, and distillation and inference-time scaling have been decoupling training compute from deployed capability for two years. Voluntary frameworks score, on their authors' own best showing, about a third of a defensible standard.

11

The case against this paper

Four arguments that cut against the conclusions above, stated as strongly as I can put them.

A paper that only marshals evidence for its own thesis is an advocacy document. Here are the four strongest arguments against what I have written, and what I think survives each.

One: automation bias may not be about machines at all

Three pre-registered Dutch experiments, the largest of them on 1,345 actual civil servants, found no evidence for automation bias. What they found was selective adherence to advice that matched a group stereotype, with no significant difference between algorithmic advice and human expert advice. If that is right, the whole framing of section 02 is aimed at the wrong target, and the remedy is not AI literacy but ordinary bias control in advice-taking.

What survives: the five tests do not depend on automation bias being machine-specific. Whether people over-defer to machines or to advice generally, they still cannot evaluate a tool they were adopted to be worse than, still cannot report a silent reweighting of their own judgment, and are still the nearest person to the failure. And there is a detail in that same research worth noticing: the study that found no selective adherence ran shortly after a national scandal, and the authors attribute the change to heightened awareness among bureaucrats. A public scandal measurably changed reviewer behaviour. That is evidence for the accountability ecosystem argument, not against it.

Two: this is all compliance cost that entrenches the incumbent

The strongest version is not the lobbying version. It is that every mechanism recommended here has a fixed cost and therefore a regressive incidence, that the firms best able to absorb it are the ones with the most to answer for, and that a compliance market creates a constituency for its own rules. That last one is not hypothetical: two of the four plaintiffs suing to preserve the US disparate-impact standard in credit are algorithmic fairness vendors whose products exist to serve it. That is not corruption. It is a predictable equilibrium, and it weakens the political case for the rule at exactly the moment it needs strengthening.

What survives: the six instruments in section 10 are unusually cheap. An override log is a database column. A named certification is a signature. A bounded-immunity reporting channel is one institution for an entire economy. The expensive parts of AI governance are the parts with the least evidence behind them, which is an argument for reallocating the spend rather than increasing it.

Three: ex-post liability beats all of this

The United States is running this experiment now, more or less deliberately. Remove the ex-ante machinery, let courts assign liability case by case, and let insurance price the residual. It has real advantages: courts see actual harms rather than hypothesised ones, they do not need a definition of a model, and they cannot be switched off by a change of administration.

What survives, and this is the part that worries me most: the insurance layer that ex-post liability depends on is going the wrong way. In November 2025 three major US carriers sought regulatory permission to write policies excluding AI liability, while a Lloyd's coverholder was already selling affirmative cover naming "AI agent mistakes, failure to escalate" as a peril. Carriers cannot price what they cannot observe, and there is no incident base rate, no severity distribution and no agreed loss trigger for an autonomous agent. So the rational move is to exclude first. The consequence is regressive: uninsured tail risk lands on the deploying company's balance sheet, where a finance director manages it by not deploying or by not documenting, and neither improves safety. Ex-post liability without a functioning insurance market is not a regime. It is a lottery.

Four: it is early, and the delays are prudent

Europe delayed its high-risk regime because the standards did not exist. Korea granted a grace period. The US agencies carved generative AI out of model risk guidance and said they would come back with a request for information. Each of those is defensible on its own terms, and rushing a regime whose compliance infrastructure does not exist produces exactly the null compliance documented in section 07.

What survives: this is the best argument on the list, and it has an expiry date. The promised US request for information has not been published. The EU's Scientific Panel was constituted ten months after the obligations it advises on became applicable. Two thirds of Member States missed the designation deadline by more than ten months. "Not yet" is a reasonable answer once. Repeated across three years and four jurisdictions, it is no longer a schedule. It is a position.

12

For the operator: the first 90 days

Nothing here needs a new regulation, a new institution, or the model to be explainable.

You are running AI in a real business. The compliance deadline that was going to force your hand just moved sixteen months. Here is what I would actually do, in order, and roughly what each step costs.

Days 1 to 30: find out what is true

  1. 01Build the inventory, and copy the two fields nobody copies. Not a list of tools. A list of decisions the tools participate in, each with a named individual and an expiry date, meaning the point at which the deployment must be re-justified rather than silently renewed. An inventory without a named owner is a spreadsheet. An inventory without expiry dates rots quietly.
  2. 02Score every consequential system on the five tests. One afternoon each. Do not delegate this to the team that built the system, for the same reason you do not let the model developer be the sole source on model quality.
  3. 03Ask every oversight function for its change log. Show me the instances where a review changed a model or its use. This is the single sharpest question in the paper and it is free. A function that has never changed anything has been captured, whatever the org chart says.
  4. 04Find the integrator. Someone in your company granted an agent a credential, wired it to a tool, and decided whether it can write to production. That person is the highest-leverage accountable party you have and they are almost certainly not in your governance structure.

Days 31 to 60: instrument before you legislate

  1. 01Turn on the override log. For every system with a human in the loop, record the recommendation, the human's decision, the time taken and the outcome where one is observable. This is a database column and a dashboard. It converts your Proof score from zero to something.
  2. 02Read the first month of it honestly. A zero override rate is a broken control, not a good model. An override rate that never changes the system is theater. A review that averages four seconds is a click.
  3. 03Run one measurement that nobody runs: system alone against system plus reviewer, on the same sample. If the reviewer is not adding accuracy, you have learned something worth more than any policy document, and you now have a choice between fixing the oversight and being honest that it is a formality.
  4. 04Put confirmation gates on every agent write. No write to a production system, no outbound message to a real person, no movement of money without an explicit confirmation. This is the control that would have prevented every documented agentic incident in this paper and it costs a day of engineering.

Days 61 to 90: make it stick

  1. 01Name one person per layer and write it down. A statement of responsibilities per named individual, one page, countersigned. Copy the design point that makes it work: you decide which risks must have an owner, not the team. And delegation does not transfer accountability.
  2. 02Give the reviewers standing. Explicit authority to restrict use, not to file a finding. If they need a manager's sign-off to say no, they do not have Power and your score is wrong.
  3. 03Fix the incentives, because this is where it decays. If the person reviewing the model reports to the person shipping the model, and their bonus depends on shipping, independence is an org chart fact rather than a behavioural one. Tie some part of the review function's evaluation to the quality and critical unbiasedness of its reviews. The 2011 banking guidance understood this and the 2026 revision quietly dropped it.
  4. 04Stand up an internal channel with bounded immunity. Anyone can report an AI failure, no sanction for good-faith reporting of an inadvertent mistake, explicit exclusion for deliberate acts and misconduct, and the channel is held by someone who cannot discipline the reporter. This is the aviation design compressed to one company, and it is the cheapest source of the failure data you will otherwise never see.
13

For the regulator: nine design rules

Drawn from what survived contact with a hostile political cycle, and from what other industries built.

  1. 01Write operational requirements, not principles. The clearest natural experiment in this paper: in 2026 the US eliminated disparate-impact liability in credit by rulemaking and did not touch the adverse-action notice rule beside it. The transparency duty survived because it is written as a hard requirement with a deadline, a form and two named insufficient answers. The fairness standard depended on a contested legal theory. Rules with deadlines and forms survive. Principles for regulators to weigh do not.
  2. 02Do not let the regulated party define its own coverage. New York's bias audit law turns on whether a tool "substantially assists" a decision, which is an unobservable internal process the employer characterises. Illinois' reporting duty attaches only where an employer relies "solely" on AI. Four years, zero filings. Trigger on the capability of the system, as German co-determination law does, not on the deployer's account of how it uses it.
  3. 03Require the oversight to be measured, and make the measurement the filing. Not an audit report on a company website in an arbitrary format. An override rate, an agreement rate, a time-on-task figure and an outcome comparison, filed with the regulator on a schedule. Every failure documented in this paper would have been visible in that data years earlier.
  4. 04Mandate publication of findings, because publication is the active ingredient. The audit that produced measurable change produced it by naming companies publicly. Peers with the same benchmark and the same published method available did not improve. If audit findings go to the company and stop there, you have funded a compliance industry, not an accountability mechanism.
  5. 05Separate the reporting channel from the sanctioning authority, and bound the immunity. Aviation gets over two million confidential reports because the custodian has no enforcement power, immunity is real, and immunity explicitly excludes deliberate acts, crimes, accidents and competency failures. AI's incident record is assembled from journalism. That is a design choice you can reverse for the cost of one institution.
  6. 06Name a natural person, and let the certification be of a process. The design insight from securities law is that you do not ask an executive to review every output, which is impossible. You ask them to warrant that a system of controls exists and produces reliable output, and you attach personal consequence to the warranty. This closes the forward-looking responsibility gap that no disclosure statute touches, and it needs no new theory of liability.
  7. 07Regulate the party with the information, not the party you can reach. Nearly half of UK financial firms admit only partial understanding of the AI they use, largely because it is third-party. You cannot close an information asymmetry by putting a duty on the side without the information. The structural answer already exists in EU financial regulation: designate the critical provider and supervise it directly. Where that is too heavy, convene pooled audits, because no single deployer has the leverage to extract audit rights from a frontier lab and a consortium might.
  8. 08Require instance-level identity and logs above a consequence threshold. Moving money, contacting real people, writing to production. Not because logs prevent harm, but because without attribution across the trace every other duty you write is unenforceable. Nobody knows whom to investigate, and nobody knows whom to call to shut it down.
  9. 09Constrain the decision, not the technology. Rules that name AI are fragile, because they can be repealed as AI policy and they rest on whatever executive posture created them. Rules that constrain a decision survive, because they do not depend on anyone agreeing what counts as AI. In US healthcare the AI-named guardrail was dropped and the technology-agnostic rule beside it, a mandatory list of inputs plus a named clinical reviewer, survived. The UK Consumer Duty never mentions a model, so it cannot be evaded by arguing about what counts as one. When the artefact is unstable, regulate the outcome.
14

For the board: seven questions

Each one is answerable in a sentence. The failure to answer is the finding.

  1. 01Who is the named individual for our highest-consequence AI decision, and what happens to them if it goes wrong? If the answer is a committee, a policy, or three names, the answer is nobody.
  2. 02Show me the override log. How often does the human disagree with the model, and what changed as a result? Zero overrides is a broken control, not a good model.
  3. 03Has anyone measured the reviewer plus the system against the system alone? If not, we do not know whether our oversight adds accuracy or only cost.
  4. 04Which of our AI systems could take an action without a human confirmation, and what is the largest thing one of them could do in an hour? That is the real blast radius, and it is an engineering fact rather than a policy question.
  5. 05If our monitoring is itself automated, is a falling alert count good news or a warning? It is a warning. Ask what would tell us the difference.
  6. 06What did we decide not to deploy in the last year, and who decided it? An oversight function that has never stopped anything has not been tested.
  7. 07What would we have to disclose if this failed publicly, and do we have it now? If the answer requires a reconstruction project, we do not have attribution and every other control is unverifiable.

One framing point for directors. Delaware's oversight doctrine has not changed for AI and does not need to. What has changed is that red flags used to be human-detectable and are now algorithmically mediated. Reliance on experts remains permissible. Blind reliance does not. And the disclosure numbers suggest most boards are at the blind end: roughly 31.6% of the S&P 500 disclose some board oversight of AI, and 1.6% specify the structure. Boilerplate is cheap. A named committee with a charter is an exhibit.

15

Open problems

Seven things I could not resolve, stated as research questions rather than opinions.

  1. 01Is there any deployed system anywhere whose human oversight has been measured against the system-alone baseline and shown to add accuracy? I could not find one published. If it exists it is the most important missing data point in the field. If it does not exist, that absence is itself the finding.
  2. 02What is the right unit of the model inventory when the artefact is unstable? A prompt edit, a temperature change, a retrieval refresh and a silent provider weight update all change behaviour without a redevelopment event. Read strictly, the inventory is uncountable. Read loosely, it is fiction. Nobody has resolved this and every framework built on a versioned artefact inherits the problem.
  3. 03Do post-hoc explanations satisfy a legal explainability duty? US credit rules require that stated reasons accurately describe the factors actually considered or scored. A surrogate explanation is by definition an approximation of the model rather than the model. No court and no rule has closed that gap, and a large compliance industry is built on the assumption that it closes.
  4. 04Can automation bias be designed around at scale? Cognitive forcing functions work and users rate them worst. That means they will not survive product iteration and must be mandated. Whether a mandated friction survives contact with a competitive market is untested.
  5. 05Who is liable when two correctly-behaving agents produce a harmful interaction? The law's unit of analysis is one system. The harm's unit is a population of interacting systems. Every allocation rule in this paper, including mine, handles this badly.
  6. 06Will an insurance market for AI harm form without a mandate? The 2025 and 2026 movement was toward exclusion. If it does not form, ex-post liability is not a regime, and the case for ex-ante rules gets stronger by default rather than by argument.
  7. 07Does a bounded-immunity reporting channel work when the reporter is a company rather than a pilot? Aviation's design assumes an individual professional with a licence to protect. The AI analogue may need to run to individual engineers rather than to firms, which raises whistleblower questions the safety-reporting literature has not had to answer.
16

Method, limits, and how this paper is maintained

How it was built

Ten parallel research passes, one per domain, each instructed to work from primary sources and to quarantine anything it could not verify. Statutes were read from the text where the host permitted it, counts were computed from published registers and datasets rather than taken from reporting, and each pass produced an explicit contested-and-unverified list that was excluded from this paper's body. Several widely repeated figures did not survive that process and are absent here as a result.

What this paper is not

  • Not legal advice. Every regime described here is moving. Check the current text before relying on any of it.
  • Not neutral about evidence quality. Where the literature is contested, section 11 says so. Where a number comes from a vendor survey rather than a primary source, I have either said so or left it out.
  • Not a claim that oversight never works. The defensible claim is narrower: oversight is unreliable, context-dependent, and almost never measured, and no policy anywhere currently requires it to be measured.
  • Not complete on scope. Defence and autonomous weapons, autonomous vehicles, platform and content regulation, and the economics of compliance each deserve the treatment that finance, health and employment get here. They are queued rather than covered, and the paper says so instead of gesturing at them.

Why there is an Update this button on this page

Roughly a third of the factual claims in this paper have a shelf life measured in weeks. Between the start of the research and the day it was published, a European regulation moved a compliance date by sixteen months, a US banking regulator rescinded the fifteen-year-old framework this paper spends a section on, and a state repealed the only AI duty of care in the country. A paper about a moving target that is not itself maintained becomes quietly wrong, which is the exact failure this paper criticises in others.

So the button files a real job. It re-verifies every dated claim against its named primary source, folds in what has moved, rewrites the affected prose rather than appending a footnote, bumps the version, and republishes with a changelog line naming what actually changed. If nothing material moved, it says so, which is also a result. The version stamp at the top of this page is the honest answer to how current it is.

References

Sources

96 sources. Every dated claim in this paper resolves to a primary source below. Where a source could not be verified, the text says so.

  1. [S1]Bainbridge, L. (1983). Ironies of Automation. Automatica 19(6), 775-779. https://ckrybus.com/static/papers/Bainbridge_1983_Automatica.pdf
  2. [S2]Green, B. (2022). The flaws of policies requiring human oversight of government algorithms. Computer Law and Security Review 45, 105681. https://www.benzevgreen.com/22-clsr/
  3. [S3]Green, B. and Chen, Y. (2019). Disparate Interactions: An Algorithm-in-the-Loop Analysis of Fairness in Risk Assessments. FAT* 2019. http://www.benzevgreen.com/wp-content/uploads/2019/02/19-fat.pdf
  4. [S4]Green, B. and Chen, Y. (2021). Algorithmic Risk Assessments Can Alter Human Decision-Making Processes in High-Stakes Government Contexts. CSCW 2021. https://www.benzevgreen.com/21-cscw/
  5. [S5]Parasuraman, R. and Manzey, D. (2010). Complacency and Bias in Human Use of Automation: An Attentional Integration. Human Factors 52(3), 381-410. https://doi.org/10.1177/0018720810376055
  6. [S6]Bansal, G. et al. (2021). Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance. CHI 2021. https://arxiv.org/abs/2006.14779
  7. [S7]Bucinca, Z., Malaya, M.B. and Gajos, K.Z. (2021). To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI. CSCW 2021. https://arxiv.org/abs/2102.09692
  8. [S8]Lyell, D. et al. (2017). Automation bias in electronic prescribing. BMC Medical Informatics and Decision Making. https://doi.org/10.1186/s12911-017-0425-5
  9. [S9]Gaube, S. et al. (2021). Do as AI say: susceptibility in deployment of clinical decision-aids. npj Digital Medicine. https://doi.org/10.1038/s41746-021-00385-9
  10. [S10]Alon-Barkat, S. and Busuioc, M. (2022). Human-AI Interactions in Public Sector Decision Making: Automation Bias and Selective Adherence. JPART. https://doi.org/10.1093/jopart/muac007
  11. [S11]Elish, M.C. (2019). Moral Crumple Zones: Cautionary Tales in Human-Robot Interaction. Engaging Science, Technology, and Society 5, 40-60. https://estsjournal.org/index.php/ests/article/download/260/177
  12. [S12]NTSB (2019). Collision Between Vehicle Controlled by Developmental Automated Driving System and Pedestrian, Tempe, Arizona. HAR-19/03. https://www.ntsb.gov/investigations/AccidentReports/Reports/HAR1903.pdf
  13. [S13]Becker, J., Rush, N., Barnes, E. and Rein, D. (2025). Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity. METR. https://arxiv.org/abs/2507.09089
  14. [S14]Kwa, T. et al. (2025). Measuring AI Ability to Complete Long Tasks. METR. https://arxiv.org/abs/2503.14499
  15. [S15]Mosier, K.L., Skitka, L.J., Heers, S.T. and Burdick, M.D. (1998). Automation Bias: Decision Making and Performance in High-Tech Cockpits. IJAP 8(1). https://doi.org/10.1207/s15327108ijap0801_3
  16. [A1]Nissenbaum, H. (1996). Accountability in a Computerized Society. Science and Engineering Ethics 2(1). https://nissenbaum.tech.cornell.edu/papers/accountability.pdf
  17. [A2]Cooper, A.F., Moss, E., Laufer, B. and Nissenbaum, H. (2022). Accountability in an Algorithmic Society. FAccT 2022. https://arxiv.org/abs/2202.05338
  18. [A3]Santoni de Sio, F. and van den Hoven, J. (2018). Meaningful Human Control over Autonomous Systems: A Philosophical Account. Frontiers in Robotics and AI 5:15. https://www.frontiersin.org/journals/robotics-and-ai/articles/10.3389/frobt.2018.00015/full
  19. [A4]Santoni de Sio, F. and Mecacci, G. (2021). Four Responsibility Gaps with Artificial Intelligence. Philosophy and Technology. https://research.tudelft.nl/en/publications/four-responsibility-gaps-with-artificial-intelligence-why-they-ma/
  20. [A5]Sharkey, C. (2024). A Products Liability Framework for AI. SSRN. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4773874
  21. [A6]Choi, B. (2019). Crashworthy Code. Washington Law Review 94(1), 39. https://digitalcommons.law.uw.edu/wlr/vol94/iss1/3/
  22. [A7]Ben-Shahar, O. and Logue, K. (2012). Outsourcing Regulation: How Insurance Reduces Moral Hazard. Michigan Law Review 111(2), 197. https://repository.law.umich.edu/mlr/vol111/iss2/2/
  23. [A8]Matera, P. (2026). Corporate Oversight in the Age of Artificial Intelligence. CLS Blue Sky Blog, 10 March 2026. https://clsbluesky.law.columbia.edu/2026/03/10/corporate-oversight-in-the-age-of-artificial-intelligence/
  24. [E1]Raji, I.D. and Buolamwini, J. (2019). Actionable Auditing: Investigating the Impact of Publicly Naming Biased Performance Results of Commercial AI Products. AIES 2019. https://dam-prod.media.mit.edu/x/2019/01/24/AIES-19_paper_223.pdf
  25. [E2]Costanza-Chock, S., Harvey, E., Raji, I.D., Czernuszenko, M. and Buolamwini, J. (2022). Who Audits the Auditors? FAccT 2022. https://arxiv.org/abs/2310.02521
  26. [E3]Casper, S. et al. (2024). Black-Box Access is Insufficient for Rigorous AI Audits. FAccT 2024. https://arxiv.org/abs/2401.14446
  27. [E4]Bean, A.M. et al. (2025). Measuring what Matters: Construct Validity in Large Language Model Benchmarks. https://arxiv.org/abs/2511.04703
  28. [E5]Ren, R. et al. (2024). Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress? NeurIPS 2024. https://arxiv.org/abs/2407.21792
  29. [E6]Longpre, S. et al. (2024). A Safe Harbor for AI Evaluation and Red Teaming. ICML 2024. https://arxiv.org/abs/2403.04893
  30. [E7]Longpre, S. et al. (2025). In-House Evaluation Is Not Enough: Towards Robust Third-Party Flaw Disclosure for General-Purpose AI. https://arxiv.org/abs/2503.16861
  31. [E8]UK AI Security Institute and US AISI (2024). Pre-deployment evaluation of OpenAI's o1 model. https://www.aisi.gov.uk/work/pre-deployment-evaluation-of-openais-o1-model
  32. [E9]NASA Aviation Safety Reporting System, Program Briefing (report volumes and immunity design under FAA AC 00-46F). https://asrs.arc.nasa.gov/docs/ASRS_ProgramBriefing.pdf
  33. [E10]Future of Life Institute, AI Safety Index, Summer 2026 edition. https://futureoflife.org/ai-safety-index-summer-2026/
  34. [E11]SaferAI, Risk Management Ratings tracker. https://tracker.safer-ai.org/
  35. [E12]OECD AI Incidents and Hazards Monitor. https://oecd.ai/en/incidents
  36. [EU1]Regulation (EU) 2024/1689 (AI Act), Article 14, human oversight. https://artificialintelligenceact.eu/article/14/
  37. [EU2]Regulation (EU) 2024/1689, Article 26, obligations of deployers. https://artificialintelligenceact.eu/article/26/
  38. [EU3]Regulation (EU) 2024/1689, Article 25, responsibilities along the AI value chain. https://artificialintelligenceact.eu/article/25/
  39. [EU4]Regulation (EU) 2024/1689, Article 43, conformity assessment. https://artificialintelligenceact.eu/article/43/
  40. [EU5]Regulation (EU) 2026/1744 (Digital Omnibus on AI), amending Regulation (EU) 2024/1689. https://eur-lex.europa.eu/eli/reg/2026/1744/oj/eng
  41. [EU6]European Commission, Regulatory framework for AI, implementation timeline and enforcement framework. https://digital-strategy.ec.europa.eu/en/policies/regulatory-framework-ai
  42. [EU7]Directive (EU) 2024/2853 on liability for defective products (revised Product Liability Directive). https://eur-lex.europa.eu/eli/dir/2024/2853/oj
  43. [EU8]Bird & Bird (2025). Proposed EU AI liability rules withdrawn. https://www.twobirds.com/en/insights/2025/proposed-eu-ai-liability-rules-withdrawn
  44. [EU9]Directive (EU) 2024/2831 on improving working conditions in platform work, Chapter III, algorithmic management. https://eur-lex.europa.eu/legal-content/EN/TXT/HTML/?uri=OJ:L_202402831
  45. [EU10]CEN-CENELEC (2025). Exceptional measures on AI standardisation, 23 October 2025. https://www.cencenelec.eu/news-events/news/2025/brief-news/2025-10-23-ai-standardization/
  46. [EU11]artificialintelligenceact.eu, National implementation plans tracker. https://artificialintelligenceact.eu/national-implementation-plans/
  47. [US1]Board of Governors of the Federal Reserve System, SR 26-2, Revised Guidance on Model Risk Management, 17 April 2026. https://www.federalreserve.gov/supervisionreg/srletters/SR2602.pdf
  48. [US2]SR 11-7 / OCC Bulletin 2011-12, Supervisory Guidance on Model Risk Management (as adopted by FDIC, FIL-22-2017). https://www.fdic.gov/news/news/financial/2017/fil17022a.pdf
  49. [US3]12 CFR 1002.9, Equal Credit Opportunity Act, Regulation B, notifications. https://www.law.cornell.edu/cfr/text/12/1002.9
  50. [US4]Executive Order 14281, Restoring Equality of Opportunity and Meritocracy, 23 April 2025. https://www.federalregister.gov/documents/2025/04/28/2025-07378/restoring-equality-of-opportunity-and-meritocracy
  51. [US5]Executive Order, Eliminating State Law Obstruction of National Artificial Intelligence Policy, 11 December 2025. https://www.whitehouse.gov/presidential-actions/2025/12/eliminating-state-law-obstruction-of-national-artificial-intelligence-policy/
  52. [US6]Colorado SB 26-189, Automated Decision-Making Technology. https://leg.colorado.gov/bills/sb26-189
  53. [US7]Texas HB 149, Responsible Artificial Intelligence Governance Act. https://capitol.texas.gov/tlodocs/89R/billtext/html/HB00149F.HTM
  54. [US8]California SB 53, Transparency in Frontier Artificial Intelligence Act. https://leginfo.legislature.ca.gov/faces/billTextClient.xhtml?bill_id=202520260SB53
  55. [US9]18 U.S.C. 1350, Failure of corporate officers to certify financial reports. https://www.law.cornell.edu/uscode/text/18/1350
  56. [US10]49 U.S.C. 1154(b), Discovery and use of NTSB reports. https://www.law.cornell.edu/uscode/text/49/1154
  57. [US11]US House Committee on Transportation and Infrastructure (2020). Final Committee Report: The Design, Development and Certification of the Boeing 737 MAX. https://democrats-transportation.house.gov/imo/media/doc/2020.09.15%20FINAL%20737%20MAX%20Report%20for%20Public%20Release.pdf
  58. [US12]Graham, D.J. (2004). Testimony before the US Senate Committee on Finance on Vioxx, 18 November 2004. https://www.finance.senate.gov/imo/media/doc/111804dgtest.pdf
  59. [US13]NHTSA Standing General Order 2021-01 incident report data files. https://static.nhtsa.gov/odi/ffdd/sgo-2021-01/
  60. [W1]Wright, L. et al. (2024). Null Compliance: NYC Local Law 144 and the Challenges of Algorithm Accountability. FAccT 2024. https://arxiv.org/abs/2406.01399
  61. [W2]Groves, L., Metcalf, J., Kennedy, A., Vecchione, B. and Strait, A. (2024). Auditing Work: Exploring the New York City algorithmic bias audit regime. FAccT 2024. https://arxiv.org/abs/2402.08101
  62. [W3]Office of the New York State Comptroller (2025). NYC DCWP: Enforcement of Local Law 144, Report 2024-N-6, 2 December 2025. https://www.osc.ny.gov/files/state-agencies/audits/pdf/sga-2026-24n6.pdf
  63. [W4]Hertel-Fernandez, A. (2024). Estimating the prevalence of automated management and surveillance technologies at work. Washington Center for Equitable Growth. https://equitablegrowth.org/research-paper/estimating-the-prevalence-of-automated-management-and-surveillance-technologies-at-work-and-their-impact-on-workers-well-being/
  64. [W5]EEOC v. iTutorGroup, Inc., consent decree, September 2023. https://www.eeoc.gov/newsroom/itutorgroup-pay-365000-settle-eeoc-discriminatory-hiring-suit
  65. [W6]Treasury Board of Canada Secretariat, Directive on Automated Decision-Making. https://www.tbs-sct.canada.ca/pol/doc-eng.aspx?id=32592
  66. [W7]UK Government, Algorithmic Transparency Recording Standard, published records. https://www.gov.uk/algorithmic-transparency-records
  67. [W8]National Audit Office (2024). Use of artificial intelligence in government. https://www.nao.org.uk/reports/use-of-artificial-intelligence-in-government/
  68. [W9]Lighthouse Reports, MIT Technology Review and Trouw (2025). Inside Amsterdam's high-stakes experiment to create fair welfare AI. https://www.technologyreview.com/2025/06/11/1118233/amsterdam-fair-welfare-ai-discriminatory-algorithms-failure/
  69. [H1]FDA, Artificial Intelligence-Enabled Medical Devices list. https://www.fda.gov/medical-devices/software-medical-device-samd/artificial-intelligence-enabled-medical-devices
  70. [H2]Wong, A. et al. (2021). External Validation of a Widely Implemented Proprietary Sepsis Prediction Model in Hospitalized Patients. JAMA Internal Medicine 181(8), 1065-1070. https://jamanetwork.com/journals/jamainternalmedicine/fullarticle/2781307
  71. [H3]Ostermayer, D.G. et al. (2024). External validation of the Epic sepsis predictive model in 2 county emergency departments. JAMIA Open 7(4). https://doi.org/10.1093/jamiaopen/ooae133
  72. [H4]Chouffani El Fassi, S. et al. (2024). Not all AI health tools with regulatory authorization are clinically validated. Nature Medicine 30(10), 2718-2720. https://doi.org/10.1038/s41591-024-03203-3
  73. [H5]FDA (2022). Clinical Decision Support Software: Guidance for Industry and FDA Staff. https://www.fda.gov/media/109618/download
  74. [H6]Finlayson, S.G. et al. (2021). The Clinician and Dataset Shift in Artificial Intelligence. New England Journal of Medicine 385(3), 283-286. https://doi.org/10.1056/NEJMc2104626
  75. [H7]Sendak, M.P. et al. (2024). Strengthening the use of artificial intelligence within healthcare delivery organizations. JAMIA. https://doi.org/10.1093/jamia/ocae119
  76. [G1]Chan, A. et al. (2024). Visibility into AI Agents. FAccT 2024. https://arxiv.org/abs/2401.13138
  77. [G2]Chan, A. et al. (2024). IDs for AI Systems. https://arxiv.org/abs/2406.12137
  78. [G3]Hammond, L. et al. (2025). Multi-Agent Risks from Advanced AI. Cooperative AI Foundation. https://arxiv.org/abs/2502.14143
  79. [G4]Willison, S. (2025). The lethal trifecta for AI agents. https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/
  80. [G5]Meta (2025). Agents Rule of Two: A Practical Approach to AI Agent Security. https://ai.meta.com/blog/practical-ai-agent-security/
  81. [G6]Model Context Protocol, Security Best Practices specification. https://modelcontextprotocol.io/specification/2025-06-18/basic/security_best_practices
  82. [G7]Brave (2025). Unseeable prompt injections in screenshots and agentic browsing. https://brave.com/blog/comet-prompt-injection/
  83. [G8]Hooker, S. (2024). On the Limitations of Compute Thresholds as a Governance Strategy. https://arxiv.org/abs/2407.05694
  84. [G9]Shavit, Y. et al. (2023). Practices for Governing Agentic AI Systems. OpenAI. https://openai.com/index/practices-for-governing-agentic-ai-systems/
  85. [G10]OECD (2026). The Agentic AI Landscape and Its Conceptual Foundations. https://www.oecd.org/en/publications/the-agentic-ai-landscape-and-its-conceptual-foundations_396cf758-en.html
  86. [I1]MeitY (2025). India AI Governance Guidelines. https://www.pib.gov.in/PressReleasePage.aspx?PRID=2228315
  87. [I2]SEBI (Intermediaries) (Amendment) Regulations, 2025, Regulation 16C. https://www.sebi.gov.in/legal/regulations/feb-2025/securities-and-exchange-board-of-india-intermediaries-amendment-regulations-2025_92066.html
  88. [I3]Reserve Bank of India (2025). Report of the Committee on Framework for Responsible and Ethical Enablement of AI (FREE-AI). https://www.rbi.org.in/Scripts/PublicationReportDetails.aspx?UrlPage=&ID=1289
  89. [I4]Cyberspace Administration of China, Measures for Labeling AI-Generated Content (translation). https://www.chinalawtranslate.com/en/ai-labeling/
  90. [I5]Cyberspace Administration of China, Interim Measures for the Management of Generative AI Services (analysis). https://www.whitecase.com/insight-our-thinking/ai-watch-global-regulatory-tracker-china
  91. [I6]Korea, Act on the Development of Artificial Intelligence and the Establishment of a Foundation for Trust (analysis). https://fpf.org/blog/south-koreas-new-ai-framework-act-a-balancing-act-between-innovation-and-regulation/
  92. [I7]Japan, Act on Promotion of Research and Development and Utilization of AI-Related Technologies (analysis). https://www.whitecase.com/insight-alert/japans-first-ai-legislation-becomes-law-focus-promoting-research-and-development-no
  93. [I8]IMDA and AI Verify Foundation, Model AI Governance Framework for Generative AI. https://aiverifyfoundation.sg/
  94. [I9]Financial Stability Board (2026). Sound Practices for Responsible Adoption of Artificial Intelligence, 10 June 2026. https://www.fsb.org/uploads/P100626.pdf
  95. [I10]UK FCA, Senior Managers and Certification Regime. https://www.fca.org.uk/firms/senior-managers-certification-regime
  96. [I11]NIST, AI Risk Management Framework. https://www.nist.gov/itl/ai-risk-management-framework
Version history

What has changed

This paper is refreshed rather than reprinted. Each entry names what actually moved, not that an update occurred.

v1.12026-08-04
  • Section 08 rewritten around **business functions**. It previously mapped roles in the AI supply chain, which duplicated section 05 and answered a question operators were not asking. It now sorts HR, Finance, Marketing, Sales, Customer Service, Operations, Safety, Legal, Procurement, IT and the executive by what regulation actually bites each one, and by the trap each one falls into.
  • Adds the three tiers of regulatory gravity, sorted by whether the output is a decision about a specific person who cannot walk away, and the observation that governance spend follows engineering while exposure sits in HR, credit, support and marketing claims.
v1.02026-08-04
  • First published. Ten parallel research passes across region, industry and function, with every dated claim tied to a primary source and unverifiable material excluded rather than softened.
  • Introduces the five tests (Name, Sight, Power, Time, Proof), the six-level oversight ladder, and an allocation rule based on the layer of irreducible private knowledge.
  • Carries the July 2026 EU deferral of high-risk obligations to December 2027, the April 2026 rescission of SR 11-7 with generative and agentic AI carved out, and the May 2026 repeal of the Colorado AI Act.
  • Known gaps, named rather than papered over: defence and autonomous weapons, autonomous vehicles, platform and content regulation, and the economics of compliance are queued for v1.1.
This paper is maintained, not archived

Update this

AI regulation moves faster than publishing cycles. Half the dated claims below have a shelf life measured in weeks. Press this and a refresh job gets filed: every time-stamped claim goes back to its primary source, anything that moved gets rewritten, new statutes, rulings and enforcement actions since 2026-08-04 get folded in, and the paper republishes with a new version number and a changelog line saying what changed. You are reading v1.1.

Written by Rahul Jindal. Views are his own and not those of any employer. This paper is descriptive and analytical. It is not legal advice, and no regime described here should be relied on without checking its current text.