AI Agents Are Shipping. Trust Isn't Keeping Up.

2026-11-09 · Aldo Pietropaolo

Part 1 of Proof Before Production, a five-part series on how to earn trust in AI agent workflows with evidence your security, risk, and audit teams can stand behind.

AI agents are moving out of the lab. In McKinsey's 2026 State of AI survey, "Forty percent of respondents from large organizations (those with annual revenues of more than $1 billion) report scaling AI agents, up from 27 percent last year" (McKinsey, August 2026). Gartner's survey of more than 2,000 CIOs and technology executives found "71% of organizations expect to deploy agentic AI within the next 12 months, yet only 21% currently have agentic AI governance capabilities in place" (Gartner, October 2026).

The people accountable for those agents are not convinced. In Deloitte's August 2026 survey of US leaders who are at least piloting agents, "70% don’t feel they can trust and govern agents" (Deloitte, August 2026). BCG's 2026 Applied AI Index puts the same gap in one headline: "By 2030, 42% of Companies Expect to Give AI Agents Real Decision-Making Authority, Yet Only 5% Have the Controls in Place to Do So Safely" (BCG, September 2026).

So agents are being deployed faster than anyone can govern them. That is why so many agent workflows sit in pilot, wrap every output in manual review, or stay away from the work where a mistake costs real money.

The trust gap is a proof gap

Read this fall's surveys side by side and the same pattern shows up. Leaders are not saying agents are useless. They are saying they cannot show three things.

What limits an agent before it acts. In Gartner's survey of cybersecurity leaders, "Fifty-four percent of organizations have no defined approach to limit AI agent access, or rely on predefined human access" (Gartner, September 2026). EY found that "85% of senior AI executives whose organization uses agentic AI admit that at least a handful of these systems execute actions without real-time human involvement" (EY, September 2026).

What their agents are doing, and how to respond when it goes wrong. In the same EY survey, 26% of respondents at companies using agentic AI said their organization "cannot detect unauthorized AI agents operating internally." ISACA's 2026 State of Cybersecurity research found that "64 percent of enterprises have not conducted any AI-related incident response exercises," and "only eight percent of organizations indicate they conduct AI-specific response exercises regularly" (ISACA, September 2026).

That their governance holds when it is tested. EY reports that about half of respondents (47%) say their organization has skipped its AI governance process for urgent deployments, "despite 98% having formal AI governance policies in place." The same survey names "the inability to accurately trace or audit the data lineage and inputs that feed critical AI decision models (72%)" among leaders' top concerns.

None of those is a model-quality problem. A better model does not give you a limit on what an agent may do, a way to stop a run, or a record of who authorized what. Those have to exist outside the agent.

Confidence is rising faster than controls

One survey this fall points the other way, and it is worth reading closely. KPMG's third-quarter AI Pulse found "73% of leaders expressing confidence in the governance and capabilities they have in place, a steep jump from 57% last quarter." In the same release: "The share of organizations building controls into their agents alongside monitoring and evaluation procedures also fell to 30% from 43% two quarters ago, as many are relying on guardrails already in place" (KPMG, September 2026).

Confidence went up while the share building controls into their agents went down. The guardrails already in place were built for people and applications. Whether they hold for an agent that acts on its own, hands work to other agents, and changes course mid-run is something you can test, and most organizations have not.

What we see in our own research

Watchlight's research agents study companies that are likely to adopt AI agents, using public sources only: company websites, job postings, filings, press, and conference talks. Across the 202 companies they researched between October 23 and November 6, 2026:

• 101 (50%) have AI agents in production, by our research agents' assessment.
• 58 of those 101 (57%) name a regulation that applies to their business.
• 36 of those 101 (36%) showed no public sign of who owns AI security or agent governance: no hiring for it, no named leader, no stated practice.
• Public hiring for AI security and governance is picking up. The number of these companies with a dated hiring signal went from at most three in any month of the first half of 2026 to ten in each of September and October.

So more than a third of the companies already running agents in production give no public sign of who is accountable for governing them.

These companies were chosen as likely agent adopters, so they are not a random sample of all enterprises. Signals are classified by keyword from each company's research, and a missing signal means we found none in public sources, not that the company has none.

Human review is holding the line, for now

Without that proof, companies fall back on people and on keeping agents away from high-stakes work. KPMG found that "Nearly half of leaders (49%) have defined high-risk use cases where they do not allow autonomous decision-making by agents."

That is a sensible response to not being able to prove anything else. It also caps what agents can do. The workflows with the most value, the ones that move money or change records, are the ones kept furthest from production, and an agent that waits on a reviewer for every action is a slower process with an AI step in the middle.

Regulators will ask for the same proof

The evidence leaders are missing is close to what the EU AI Act requires for high-risk systems. Article 12 says "High-risk AI systems shall technically allow for the automatic recording of events (logs) over the lifetime of the system." Article 14 requires that the people overseeing a system be able to intervene in its operation "or interrupt the system through a ‘stop’ button or a similar procedure that allows the system to come to a halt in a safe state." Article 26 requires deployers to keep the automatically generated logs for at least six months (Regulation (EU) 2024/1689).

The Digital Omnibus on AI, Regulation (EU) 2026/1744, moved when those obligations apply: from 2 December 2027 for high-risk systems listed in Annex III, and from 2 August 2028 for those covered by Annex I. It did not amend Articles 12, 14, or 26. The extra time is for building the evidence. Part 5 of this series covers the regulations in detail.

What earns trust

Trust in an agent workflow comes from being able to show four things before it goes to production, on that specific workflow:

1. A written authority boundary. What the agents may do, on whose behalf, with which tools and data, what is off limits, and which actions need a person's approval.
2. Tests that try to break it. Actions outside that boundary attempted on purpose, and shown to be denied before they run.
3. A stop that works on a live run. A run pushed off course, and contained while it is still active.
4. A record you can hand to a reviewer. Every governed action tied to who authorized it and which policy decided it, in a record that shows if anyone alters it.

How we help teams build that evidence

We spend our days on this problem: researching how companies put agents to work, mapping what regulators and auditors will ask for, and working through what it takes to get an agent workflow approved. Our services follow the path most teams take:

• Not sure which workflow to start with? The Agent Workflow Fit & Readiness Assessment maps up to three planned or active workflows in two to three days and tells you which one is ready to prove first.
• Need to settle how identity should work for agents? Agentic IAM Architecture designs agent identity, credentials, and delegated authority around the IAM, PAM, and IGA you already run, whichever vendors they are.
• Ready to get one workflow approved? The Production-Readiness Pilot puts all four pieces of evidence in place on one workflow in about six weeks, and ends with a go or no-go decision your security, risk, and business owners sign.
• Need to show which rules apply? The Agent Compliance Map lays out what nine frameworks, including the EU AI Act and NIST AI RMF, ask of AI agents.

Human review does not go away. It moves to where it matters: the actions your boundary marks as needing a person, instead of every output the agent produces.

The rest of the series

• Part 2: Trust is a decision. What a production go or no-go on one agent workflow needs, and who has to sign it.
• Part 3: Prove it before you ship. Testing your agents on purpose against the ways access stops being enough.
• Part 4: When it goes wrong anyway. Why being able to stop a run in progress is part of trusting it.
• Part 5: The evidence regulators will ask for. The EU AI Act, US state rules, and NIST work on agent identity, mapped to what a governed run produces.

Four questions to ask about your own agents

1. Do you have a current list of your agents and what each one is allowed to do?
2. If one of them started doing the wrong thing right now, how long would it take to stop it?
3. Could you show who authorized a specific action an agent took last week, and why it was allowed?
4. Which of your agent workflows is waiting on security sign-off, and what evidence would get it approved?

Bring us the agent workflow your security team has not signed off yet. Scope a production-readiness pilot, or request a demo and walk through how we would take it from pilot to an approved production decision.