AI agent governance: controls buyers must demand

AI agent governance is now the deciding question in agent procurement, not an afterthought. Microsoft warned in October 2026 that its own deployed agents behave like insider threats and launched Guardian Agents to watch them, citing its security chief saying frontier models cannot be trusted (TechCrunch). Days later ServiceNow gave agents cryptographically signed identities through its AI Passport, checked at runtime against what the enterprise authorised (CIO).

That pairing matters to anyone buying or selling agent work right now. Adoption has outrun oversight: 66% of organisations already run agents in limited or scaled production, according to G2's survey of more than 1,000 B2B software buyers (G2). Contracts signed this quarter will decide who carries the risk when an agent acts outside its brief.

Why AI agent governance is a procurement issue now

The governance gap is measurable. Deloitte's survey of 3,235 IT and business leaders across 24 countries found only 21% have a mature governance model for agentic AI, and around 80% lack basics such as clear decision boundaries, real-time monitoring and audit trails (Deloitte).

Incidents are already landing. In EY's survey of 202 senior AI executives, 36% reported an AI incident or failure with materially negative impact, including data loss, financial damage and operational disruption (EY).

Agents are also proliferating faster than inventories. G2 found 38% of organisations with production agents run more than 100 of them, while 47% still use informal or decentralised management (G2). Anthropic figures cited by CIO put unregistered agents in the networks of 82% of organisations (CIO). A buyer who cannot list the agents touching their systems cannot govern them.

The control gaps buyers should test for

Traditional RFP templates do not cover autonomous systems well. Feature checklists say little about how an agent handles exceptions, works across systems or stays accountable (TechTarget). Four failures recur in evaluations.

• Agents with no real identity. The agent simply asserts what it is allowed to do. As one analyst put it, taking an agent's word for its permissions is an Achilles' heel of agentic AI (CIO).
• Blank-check permissions. Broad credentials are issued once and never narrowed per task. This is why Microsoft now adjusts agent permissions dynamically and requires human approval for sensitive steps (TechCrunch).
• Build-time checks only. Static code review cannot catch prompt hijacking, poisoned context or goal drift, which show up at runtime (CIO).
• Unauditable evidence. Logs that only the vendor can read prove nothing in a dispute or a regulatory review (AAES).

Require a verifiable identity for every agent

Every agent acting in your systems should carry a verifiable identity and an owner. ServiceNow's AI Passport does this with a cryptographically signed credential, checked at runtime, that defines the authorised scope and can trigger restrictions or a kill switch on deviation (CIO).

Use that as the benchmark, not the only option. Ask how the agent authenticates, which named person is accountable for its authority, and whether credentials are tenant-isolated. The credential must not be retrievable in a way that lets the agent bypass the control (AAES).

That line from analyst Frank Dickson, quoted in CIO, is the distinction to hold vendors to. Recording is table stakes. Enforcing at the door is the requirement.

Enforce least privilege at runtime, not on paper

Least privilege for an agent is not a role in a design document. It is a live decision made on each request: which tools, accounts, destinations and data the agent may use for this task, right now.

Microsoft's approach limits agent access to data and systems and adjusts permissions by task, including human approval at defined points (TechCrunch). ServiceNow enforces passport scope at runtime, redirecting, restricting or reconnecting an agent that deviates (CIO).

Test it rather than read about it. An over-provisioned agent, such as an email agent with 50 tools attached, is a red flag worth investigating before go-live (CIO). Ask for negative tests that show the control refusing an action it should refuse.

Demand runtime monitoring and an agent inventory

AI agent governance starts with visibility. You need one record per agent across every platform it runs on, because each platform sees only its own slice. An agent that reads customer records in one cloud, holds a payments credential in another and can send messages from a SaaS tool forms an attack path no single console detects (CIO).

Detection of unapproved agents matters as much as controlling the approved ones. Microsoft added agent discovery to Defender, covering up to 60 types of agents employees bring in (TechCrunch). ServiceNow added shadow-AI detection and more than 400 connectors for AI discovery (CIO).

The need is real: 26% of executives whose organisations use agentic AI admit they cannot detect unauthorised agents operating internally (EY).

Price the oversight, not just the licence

Oversight has a running cost, and buyers should budget for it before scaling. G2 calls this the distrust tax: 31% of B2B buyers say employees spend six or more hours a week bot-sitting, and 45% say oversight and corrections consume more than a quarter of expected productivity gains (G2).

Ask vendors what the budget meter measures, what is reserved before an agent executes a spend action, and whether concurrent requests can exceed the ceiling (AAES). Ask too what happens when the control plane fails: do old grants still work, and for how long.

Recovery questions deserve equal weight. Find out what survives restore, rollback and migration, and whether you can verify audit records without the vendor's service. These are the terms that decide your exposure after a bad day.

What vendors should build before the next RFP

Vendors pitching agent work will face these questions from now on. Analysts already flag that loose permissions handed to capable agents amount to a blank check for doing harm (CIO). Building the answers in is cheaper than losing the deal.

1. 1Ship a signed agent identity tied to a named human owner, verifiable at runtime and usable across third-party agents.
2. 2Scope every task narrowly and enforce scope at execution, not at design time.
3. 3Make approvals bound to the exact request, with expiry and clear revocation.
4. 4Export tamper-evident audit records the buyer can verify independently.
5. 5Publish negative tests and version-specific evidence, and say plainly what you only observe rather than block.
6. 6Price oversight and exception handling openly, so buyers can plan the run cost.

Buyers shortlisting firms can browse the cybersecurity vendor directory for specialist help, or read the AI agents guide before scoping work. The AI governance guide covers the policy side that sits above any single product.

Frequently asked questions

What is AI agent governance?

It is the set of controls that decide what an agent may do, under whose authority, and how that activity is recorded and stopped. In practice it combines verifiable agent identity, least-privilege scope enforced at runtime, monitoring and tamper-evident audit trails. Without all four, oversight is a review after the fact.

Why are AI agents compared to insider threats?

Because the agents an enterprise already pays for can act unpredictably with real access to sensitive data and tools. Microsoft's security chief said frontier models cannot be trusted because they are trained to achieve tasks rather than follow organisational norms (TechCrunch). The risk comes from inside the perimeter, not only from outside attackers.

What should an agent governance clause in a contract cover?

It should name the control, the exact product version, the evidence you will receive and the test that proves the control works. Ask separately who controls signing keys and timestamps, which data leaves your environment, and what survives termination (AAES). Leave unanswered items open rather than treating silence as compliance.

Do agents need human approval for every action?

No, and requiring it erases the benefit. Boundaries should mark which decisions need a human and which do not. Set approval for irreversible or externally visible actions, and let low-risk reads run freely within scope (TechTarget).

What this means for buyers

Treat AI agent governance as a gate, not a score. Before scaling, do four things:

1. 1Write a one-page risk profile naming the business owner, permitted data, allowed actions and stop conditions for each workflow.
2. 2Turn every proposed action into a requirement with matching evidence, and keep a gap open until it is demonstrated.
3. 3Run a supervised test in your own tenant: a refused action, an expired approval, a revoked credential, a restored audit export.
4. 4Budget for oversight hours and exception handling in the business case, not just licence fees.

Where internal skills fall short, EY found 69% of executives worried about exactly that gap (EY). Bring in reviewers early, list the requirement in your project brief so vendors respond to the real bar, and compare responses side by side before signing.