Why this module matters
Traditional AI governance was designed for systems that predict things. Agentic AI is different. Agents do not just make predictions — they take actions: they read information, use tools, call APIs, send messages, trigger workflows, and even hand work to other agents. In regulated organisations, this creates new risks around control, accountability, safety, oversight and legal exposure. This module teaches you how to govern agentic systems in a safe, controlled and auditable way.
Core idea
Approval alone is not enough. A real control must also let you limit, monitor and revoke an agent's access when needed.
“Approval without revocation is not a control. It is a hope.”
— The principle that reframes this module
CH 4.1
Why agents are different from models
- Most existing AI governance was built for models that produce outputs from inputs. A model answers a question.
- An agent acts across systems. In a single task an agent may:
- read a CRM record
- draft an email
- query a vendor system
- update a ticket
- trigger a workflow
- call another agent to continue the task
- The governance challenge is therefore no longer just the model itself. It is the full chain of action across people, systems, tools and vendors.
- Agents are already appearing inside platforms such as Microsoft 365, Salesforce, ServiceNow, core banking platforms, and other enterprise SaaS.
- In many organisations, business teams are also building their own internal agents without going through formal governance.
CH 4.2
Where current governance frameworks break down
- Current AI governance frameworks do not fully fit agentic systems. They break in four main ways.
Break 01
Inventories assume fixed systems
Traditional governance assumes each AI system can be listed once and reviewed periodically. Agents compose actions dynamically at runtime, so the workflow may change during execution.
Break 02
Validation assumes stable behaviour
Many AI controls assume the system behaves consistently between reviews. Agent behaviour depends on the prompt, available tools, memory, context, permissions and previous actions.
Break 03
Accountability becomes unclear
If Agent A calls Agent B, and Agent B uses a vendor tool to complete the action, it becomes difficult to say who is responsible for the final outcome. This creates a serious accountability gap.
Break 04
Approval is not enough
With agentic systems, the more important question is whether the organisation can quickly remove access, stop action execution, and prove that the stop worked.
CH 4.3
Inventorying agents at runtime
- A static list of AI systems is not enough for agentic environments. Organisations need a live understanding of what agents exist, what they can access, and what they are doing.
- The inventory must include:
- internal agents built by business teams
- vendor-shipped agents inside enterprise platforms
- agents activated indirectly through software updates or platform changes
- For each agent, capture: tools and APIs it can use, sub-agents it can call, memory or retrieval sources, identities and credentials, systems it can affect
- Governance must shift from a one-time inventory to continuous discovery and monitoring
CH 4.4
Authorised scope, approval and revocation
- An agent should not just be described in policy. Its limits must be enforced by the system.
- Important control patterns: human approval for sensitive actions, sandboxing, budget limits, rate limits, restricted tool lists, session time limits
- Identity and access management for agents: agents need their own identities, permissions and secrets handling, following least-privilege principles
- Revocation must be a first-class control. If an agent behaves unexpectedly, the organisation should be able to disable its permissions, shut down its access, and confirm that the revocation worked.
- Target a 90-second revocation, with evidence the control fired. Tested regularly, not just documented.
CH 4.5
Accountability across agent chains
- When multiple agents and tools work together, accountability must be defined at two levels:
- who owns the agent at design time — accountable for what it is authorised to do
- who owns the agent at runtime — accountable for what it actually did in a specific session
- For every production agent, the organisation should be able to answer four questions:
- 1. Who is the named owner of the agent at design time?
- 2. Who is the named owner during runtime?
- 3. What actions is the agent allowed to take, and how is that enforced?
- 4. How can the agent be revoked, and when was that revocation last tested?
- Contracts alone do not solve the accountability problem. The organisation still needs a clear internal ownership model.
CH 4.6
Safety and examiner readiness
- Regulators and examiners are increasingly likely to ask how organisations govern agents that can take real-world actions.
- Existing rules on model risk, IT control, operational risk, and AI governance already support these oversight expectations — no new authority is required.
- The likely examiner questions:
- Can you show your full inventory of production agents?
- Can you name the individual accountable for each agent?
- Can you demonstrate how an agent is prevented from acting outside its authorised scope?
- Can you produce evidence that revocation worked the last time it was tested?
Evidence a regulator or auditor may expect
- A live inventory of agents in production
- Owner records — design-time and runtime, by name
- Scope controls enforced by the system
- Revocation records and the date of the last test
- Proof that controls were tested and that they fired correctly
◆ Capstone-feeder lab: (a) identify agents in your environment · (b) select one agent · (c) define the design-time owner · (d) define the runtime owner · (e) describe the system-enforced scope · (f) test or document the revocation path. This artefact carries directly into the Module 7 capstone.
RegTech AI Solutions
NL2SQL Analytics
HealthCare
Precision AI
Community