AI Agent Governance: Approval, Audit, and Eval Gates
Who owns the agent’s decision when the agent acted correctly according to code but wrongly for the business?
The tempting answer is to write a policy document and assume teams will remember it during runtime. That answer is not useless, but it is too vague to operate. AI agent governance is the operating system for agent risk: ownership, risk tiers, permitted actions, approval rules, audit trails, eval gates, incident review, and change control. Good governance is concrete enough to run inside the workflow, not just inside a policy document.
Direct answer
AI agent governance is the operating system for agent risk: ownership, risk tiers, permitted actions, approval rules, audit trails, eval gates, incident review, and change control. Good governance is concrete enough to run inside the workflow, not just inside a policy document.
Data note
When this matters
- Agents move from internal demos to production workflows.
- Multiple teams own prompts, tools, data, security, and final outcomes.
- A customer, auditor, or executive may ask why an agent acted.
Failure modes this page should catch
- No one owns the final action because the model, tool, and workflow owners are different teams.
- Every risk discussion happens at launch, not at runtime.
- Approval rules are inconsistent across similar actions.
- Incident review lacks traces, evals, and policy decisions.
Agent governance matrix
| Gate | Signal | Action |
|---|---|---|
| Owner | business, engineering, security, data | Name the accountable person |
| Risk tier | read, suggest, mutate, external, irreversible | Map tier to control |
| Approval | when human review is required | Make rule executable |
| Audit | trace, policy, eval, final action | Preserve evidence by turn |
| Review | incident, regression, scope change | Update gates after failures |
Running example
A support agent can draft refund language at tier 1. Issuing the refund is tier 3 because it moves money and changes customer state. Governance is the rule that prevents those two actions from sharing the same approval path.
Copy the working template
Use the agent governance matrix above as the v1 artifact for this page. Replace the placeholders with your own agent names, tools, risk classes, and thresholds, then link the result back into your monitoring, tracing, security, and evaluation gates.
How this connects to the control-gates library
- AI Agent Control Gates: Stop Bad Agents Before They Act
- AI Agent Security: Threat Models for Tool-Using Agents
- AI Agent Evaluation: Gates That Catch Bad Behavior
- Human Approval for AI Agents: When Agents Should Stop
- Agent Tracing: A Practical Schema for Tool-Using AI
Frequently Asked Questions
What is AI agent governance?
AI agent governance defines who owns an agent, what actions it may take, which risks require approval, what evidence must be logged, and how incidents change future controls.
How is governance different from guardrails?
Guardrails are checks. Governance decides which checks are required for each risk tier, who owns exceptions, and how the organization reviews failures.
What is the minimum governance system?
Start with owners, risk tiers, approval rules, trace requirements, eval gates, and an incident-review loop. Do not begin with a giant policy document.
The Takeaway
Governance works when it becomes a runtime map from risk to control, not a PDF beside the product.