When AI moves from conversation to action: governing agentic AI

Over the past few years, organisations have created AI inventories, risk classifications, approval processes, and controls for privacy, security, fairness, transparency, and human oversight. That foundation still matters. But agentic AI introduces a different governance issue.

An AI agent can access systems, pursue a goal across multiple steps, communicate with customers or suppliers, update records, initiate transactions, and coordinate with other agents. The management issue underneath this technology is familiar: how do you delegate authority without losing control?

The unit of AI governance is shifting from a heavy focus on models and use cases towards delegated authority embedded in business processes and, increasingly, networks of agents. Consider a procurement example: a generative AI assistant might analyse supplier proposals and recommend a preferred supplier. The human still decides what happens next. An agentic procurement system could go further: request quotations, interact with supplier systems, compare offers, negotiate within parameters, update the procurement platform, and initiate an order.

The underlying AI may not be dramatically different, but the organisational consequence is. A hallucinated recommendation is a reliability problem. A hallucination that results in a purchase order has a real business consequence. This is why existing AI governance cannot simply be copied and pasted onto agents.

Two dimensions specifically matter in this context: autonomy and action-space. Autonomy describes how independently an agent can decide how to pursue a goal. Action-space describes what its tools and permissions allow it to do. An agent may have considerable freedom to decide how to complete a task but only be able to read internal information. Another may follow a tightly prescribed workflow but have permission to transfer money. Both dimensions matter.

The practical implication for organisations is clear: the consequences an agent can create matter as much as the quality of what it generates.

Many organisations have introduced governance processes in which an AI use case is registered, risk-classified, assessed, and approved. That remains useful, but approving an agentic use case such as ‘customer-service agent’ does not cut it.

Imagine that the agent can retrieve customer information, change an address, cancel an order, issue a refund up to €50 independently, propose a larger refund for human approval, and terminate a contract. Those are not one governance decision. They are different delegations of authority with different consequences.

This is where a digital mandate can help. Rather than approving the agent as one package, the mandate describes what it is there to achieve, which decisions it can make, which actions it can take independently, what systems and data it can access, where limits apply, and when human involvement is required.

The key is to assign autonomy to individual decisions and actions rather than label an entire agent ‘autonomous’ or ‘human-in-the-loop’. Within the same mandate, an agent may act independently on some decisions, require approval for others, and escalate selected decisions to a person. That leads to the organisational question ‘Which decisions are we prepared to move from “AI recommends and a human decides” to “AI decides and a human oversees”?’

Accelerating agentic AI does not have to mean giving agents maximum autonomy everywhere. A more useful approach is to vary autonomy according to the consequences of an action and the organisation’s ability to detect and reverse mistakes. A practical tiering model can consider four factors: the potential impact of an incorrect action; whether the action is reversible; the sensitivity of the data or systems involved; and whether meaningful human intervention is feasible.

For example, a meeting-scheduling agent may be allowed to operate independently because mistakes are visible and reversible. A customer agent might issue small refunds autonomously but require approval above a threshold. An HR agent might schedule interviews and answer policy questions, while hiring, disciplinary, and termination decisions remain reserved for people.

This makes autonomy a management choice rather than a technical feature. It can also be increased over time. If monitoring demonstrates that an agent performs reliably within a bounded mandate, the organisation can deliberately expand that mandate. With that, governance becomes the mechanism that enables scaling rather than the mechanism that blocks it.

Earlier AI governance often uses human review as the ultimate safeguard. That becomes harder as the volume and speed of agentic activity increase. If an agent makes ten recommendations per week, an employee can genuinely review them. If multiple agents perform thousands of actions, inserting approval boxes everywhere can create the appearance of control while producing alert fatigue and automation bias. Instead of asking ‘Is there a human in the loop?’, it becomes more useful to ask: ‘At which moments does human judgement materially reduce risk?’

That may mean approval before an irreversible action, intervention when defined risk indicators are triggered, exception handling rather than routine review or retrospective sampling for low-risk activity. The effectiveness of that oversight can also be monitored through measures such as override rates, approval times, recurring exceptions, and signs that reviewers are simply accepting the agent’s recommendation.

There is an evident workforce implication here. If people increasingly supervise work they no longer perform, the organisation still needs enough expertise to recognise when the agent is wrong. A control environment that depends on expert human judgement becomes weaker if that expertise is no longer developed in practice.

Agentic workflows can make ownership less visible. A developer built the agent. IT operates the platform. A vendor supplies the model. The business owns the process. Risk defined the policy. An employee initiated the workflow. Who owns the outcome? The answer cannot be ‘the agent’. Autonomy can be delegated; organisational accountability cannot.

Each consequential agentic process can have a named human owner who understands the mandate, accepts the residual risk, can change the authority granted, and can stop the agent when necessary. That person does not need to approve every action, just as managers do not approve every action performed by employees. What senior management needs is visibility of which agents act on the organisation’s behalf, what authority they hold, who is accountable for that authority, and whether it remains appropriate.

Existing AI governance has already brought data governance into focus. Agents introduce an adjacent challenge: which organisational knowledge are they authorised to rely on when taking action?

Consider an HR agent handling an employee request. It finds a policy updated last month, an older procedure on SharePoint, a manager’s email describing an exception, and historical cases in which teams acted differently. A human employee may understand that these sources do not carry equal authority. For an agent, that hierarchy needs to be explicit in its operating environment.

This matters because the consequences of poor-quality knowledge change once AI can act. With generative AI, outdated guidance may produce a poor answer. With agentic AI, the same outdated guidance can drive hundreds of wrong actions. As AI moves from generating information to acting on it, knowledge governance becomes part of the operational control environment. This brings questions such as which sources are authoritative, who owns them, how outdated information is removed, how conflicts are resolved, and whether the organisation can reconstruct which information an agent relied on.

The governance challenge becomes more complex when agents start interacting with other agents. A procurement agent may communicate with a supplier’s sales agent. A logistics agent may coordinate with a carrier. A customer’s personal agent may interact directly with a company’s service agent. At that point, the organisation no longer controls the whole interaction. It can govern its own agent, but not necessarily the agents it encounters. As agents start communicating and transacting across organisational boundaries, governance therefore extends beyond controlling your own agent to deciding which other agents it can trust, what it can share with them and what they are allowed to agree on.

This suggests an emerging category of agent counterparty risk. When one agent transacts with another, several questions arise. Who is the other agent acting for? What is it allowed to agree to? What information can our agent share with it? Can it involve other agents? And who is responsible if something goes wrong?

A complete ‘agent network’ governance architecture is not needed today, but organisations looking to scale agents can design with the possibility that some future counterparties will be agents rather than people.

The aim of these six areas of action is not more governance, but governance that makes greater autonomy possible:

1. Agent registry. An extension of the existing AI inventory can show which agents are operating, who owns them, what tools and systems they can access, what knowledge they rely on, what authority they hold, and which external agents or services they can interact with.
2. Agent mandates. A mandate separates technical capability from organisational authority. It can capture permitted decisions, actions, limits, escalation conditions, and accountable ownership. Prompts can tell an agent what it is allowed to do. Permissions and system controls determine what it is actually able to do.
3. Autonomy tiers. Actions can be grouped by impact, reversibility, sensitivity, and uncertainty. Low-consequence actions may receive more autonomy, while higher-consequence actions can have checkpoints or remain reserved for people.
4. Human oversight. Human judgement adds the most value at points where it changes the risk, rather than at every step. Override rates, approval patterns, and recurring exceptions can show whether oversight remains meaningful and whether reviewers retain the expertise to challenge the agent.
5. Knowledge governance. Agentic action depends on which sources the agent treats as authoritative. Clear ownership and correction mechanisms help manage the operational risk created by outdated, conflicting, or incorrect data and knowledge.
6. Agent-to-agent interaction. When agents interact across organisational boundaries, identity, authentication, permissions, information sharing, logging, and counterparty controls become part of the governance design.

Governance and acceleration are often presented as competing objectives: more autonomy creates more value but more risk; more governance reduces risk but slows adoption. Agentic AI offers a different way to think about that trade-off.

If an organisation cannot distinguish between an agent that reads a document and one that can transfer money, it has little choice but to treat both cautiously. When mandates, permissions, autonomy levels, escalation paths, and accountability are explicit, organisations have a clearer basis for deciding where greater autonomy is appropriate.

The first generation of AI governance largely asked: can we trust the system and its outputs? Agentic AI adds a second question: what authority can we trust the system to exercise on our behalf? And as agents begin interacting across organisational boundaries, a third follows: how do we govern delegated authority when no single organisation controls the entire interaction?

Organisations are not starting from zero. Existing approaches to delegated authority already include mandates, approval limits, least privilege, segregation of duties, authoritative policies, supervision, audit trails, and accountability. Some translate well to agents. Others will need to evolve because agents can act at machine speed, repeat failures at scale, and interact with systems outside the organisation's control.

The objective of agentic AI governance is not to keep humans involved in every action. It is to create the conditions under which organisations can delegate authority without losing control.

The question for executives is therefore no longer only, ‘What can AI do for our organisation?’ It is: ‘What authority are we prepared to give AI - and have we designed the organisation to remain in control when we do?’