Governing High-Impact AI and Assurance
LDR-A03 · Advanced level · ~150 min
Capabilities
- C5Evaluation and Quality Assurance
- C6Responsible, Secure, and Appropriate Use
- C7Workflow and Human-Oversight Design
Source
- NIST-AI-RMF
- NIST-GAI-PROFILE
- OECD-AI-PRINCIPLES
Profiles: leader
Loading…
Activity: Three-Lens Classifier
Classify each system with the three lenses. When information is missing, choose "Cannot determine" and note what you would need. Then reveal the suggested answers and check your automation-vs-AI calls.
| System | How it operates | What it does | Where it sits |
|---|---|---|---|
| Mail merge | |||
| Spam filter | |||
| Recommendation engine | |||
| Chatbot | |||
| Rules-based approval | |||
| Tool-using agent |
Activity: AI Landscape Map
Pick one AI product you use or have observed. Fill in the eight dimensions. Then export your map — it becomes the core of your checkpoint submission.
Activity: Task Scorer
List up to ten recurring tasks. Score each 1–5 on potential value (would AI plausibly improve the net result?), risk (consequence of error, data sensitivity), and verification effort (cost to confirm the output is right). Reject at least two tasks — an honest rejection is part of the method, not a failure.
Scoring: 1 = very low · 3 = moderate · 5 = very high. For verification effort, 5 means it would cost a lot to check the output.
| Task | Value (1–5) | Risk (1–5) | Verification effort (1–5) | Decision |
|---|---|---|---|---|
The three lenses — look at any system through all three
How it operates
Rules someone wrote, patterns learned from data, or a hybrid.
What it does
The action: predict, classify, detect, recommend, generate, retrieve, act.
Where it sits
The layer: model, data source, application, connected tool, workflow, human checkpoint.
One chat box can hide all three at once. Separate them before judging the system.
Worked example — "The AI answered the ticket" is five components
Classify
request type
Retrieve
policy doc
Draft
reply
Route
rules
Human
checkpoint
Five different components, each a place where incorrect data or weak oversight can cause harm. The sentence hides all of them.
The equation that decides
Low verification cost → good first use
Summarizing your own notes. You already know what's right — checking is cheap.
High verification cost → avoid first
Summarizing unfamiliar legal terms. You'd have to verify every claim — the checking is the real work.
An unpleasant task is not automatically an AI-suitable task. Run the equation first.
Review · flashcard
Flip each card, say the answer out loud first, then check.
Your takeaway card — what to keep from this module
- 1define assurance scope, require impact and control evidence, evaluate recourse and monitoring, commission independent review, and pause or retire systems.
- 2Assurance asks whether claims, controls, and evidence are sufficient for the intended consequence.
- 3Require documented purpose, affected groups, data lineage, performance by relevant segment, limitations, human factors, security, accessibility, recourse, incident history, monitoring, and change controls.
- 4Independence matters when incentives favor launch.
- 5Deliverable: Create an Assurance Decision Memo with approve, conditionally approve, reject, or pause recommendation.
Try it yourself · Apply LDR-A03 — do the real task
This experiment takes the LDR-A03 Apply step into your own work with the AI tools you actually have.
Pick your tool (to confirm what's available — the method works with any)
Tip: open ChatGPT (free) (chatgpt.com — free tier, no login needed for most use) in a new tab to begin.
Do the task the current way — no AI. Time it, and note errors or rework. This is your baseline.
Now produce the LDR-A03 deliverable with your AI tool. Use a real prompt with your goal, context, and constraints.
Here is my task: Create an Assurance Decision Memo with approve, conditionally approve, reject, or pause recommendation.. My goal is [goal]. Context: [what the AI needs to know]. Constraints: [limits, format, tone, length]. Produce the deliverable, and tell me where you are not certain.
Verify the AI output using the module's evaluation criteria: Check independence, subgroup evidence, recourse, residual risk, named acceptance authority, monitoring, and stop power. Time the verification.
Compare baseline vs AI honestly. Run: net value = benefit − (setup + review + correction + tool + switching).
Record what you observed