Nobody got fired, they got subpoenaed
UnitedHealth Group ran an AI with a 90% error rate. On purpose.
That's what the class action says. The nH Predict algorithm, built on 6M patient records, automatically overrode physician recommended care. If patients appealed, they won 9 times out of 10. Courts are now forcing UnitedHealth to answer: Did they build governance around a model they knew was wrong? Discovery was ordered last month. It's not looking good.
Now, I don’t think anyone at UnitedHealth actually sat in a conference room and decided to hurt old people. What's more likely? They deployed a model, watched it generate good financial outcomes, and never asked whether it was actually working. Everyone was driving toward their number. And nobody's bonus was tied to accuracy. The governance layer that should have caught a 90% error rate before it touched a single patient didn't exist.
That's what ungoverned AI looks like.
Most companies aren't auto-denying claims. But the governance gap is the same: Who decides what goes to production? Who monitors live systems? Who pulls the plug?
S&P Global found that 42% of companies abandoned most of their AI initiatives last year. Not because the tech failed, because the org did. Governance isn't a compliance checkbox. It's a load-bearing wall. Here are four ways companies build it:
- The Gatekeeper -
Open idea submissions, cross-functional review, approval before prototypes. Adobe built this with risk scoring: every proposal gets rated on audience sensitivity, data exposure, and objective type. Works best for early-stage programs and regulated industries, when you have limited build capacity.
- The Center of Gravity -
A standing team owns standards, tooling, and evaluation. Business units propose, the CoE decides and co-builds. BBVA runs an internal employee competition to feed the pipeline, then monitors everything that reaches production. Works when idea volume outpaces committee bandwidth, but units can't yet self-govern.
- The Franchise Model -
Teams build within centrally defined guardrails. Central review only kicks in before production. JPMorganChase's CDO sets data and model risk standards: 450+ use complying cases got built. Scales only when the guardrails are crystal clear.
- The Mandate -
C-suite defines 3-5 investment themes. Projects that don't map to a theme don't get funded. Walmart's CTO defined a surgical agent deployment strategy: Task-bounded proposals only. Everything else dies at the idea phase. Good when the bottleneck is change capacity, not ideas.
The pattern?
Governance isn't about who reviews the work.
It's about when authority enters the process.
UnitedHealth had a model, a mandate to use it, and metrics that made the error rate invisible. They didn't have a structure that made someone responsible for asking: Is this model still right?
UnitedHealth didn't lack smart people or good intentions.
They lacked a single person whose job was to ask hard questions.
That's a cheap problem.
Until it isn't.