Failure Modes and Accountability
Why this matters
Every useful AI system will fail. The only question is whether the failure is visible, contained, and owned. A system with no named owner is not a system; it is a surprise with a schedule.
What you will be able to decide after this
- What the common failure modes look like in a real use case of yours.
- How you would know a failure was happening before it mattered.
- Who is accountable — and why it cannot be the model.
Core lesson
Common failure modes:
- Confident fabrication presented as fact
- Silent degradation when data or context drifts
- Correct-looking output that is wrong in the specific case that matters
- Loss of ownership ("the AI said so")
Accountability cannot be delegated to the model. Someone has to own the decision to use the output, the method of checking it, and the response when it is wrong.
Note the shape of these failures: they are not dramatic. Nothing crashes. The output still looks fine. That is why they survive — the failure is invisible until someone checks, and if nobody owns the checking, nobody checks.
Worked example
A support team routes tickets using an AI classifier. For eleven months, accuracy is fine. Then a product launch changes the vocabulary customers use, and the classifier starts routing new-product questions to the wrong queue. Nothing alerts anyone. The system is not failing loudly; it is degrading silently, and the team only notices when a customer escalates.
The fix is not a better classifier. The fix is the person who owns the question "how would we know if this started drifting?" — before it drifts.
Practice
For one AI use case you care about, list how it can fail. For each failure mode, write how you would know — a number, a check, a person who notices. If you cannot name the detection, name that gap.
Apply — produce the artifact
Write a one-page failure and ownership brief: how it can fail, how you will know, who is accountable, what happens next.
Verify
Ask: "If this fails on a high-stakes case next month, is it obvious who owns the response?" If the answer is fuzzy, revise.
Sources
This module is original practice guidance based on the authoring standard and does not depend on a specific external factual claim. Editorial review is still required.