A swarm of autonomous OpenAI agents spent six weeks this summer turning an obscure, 25-year-old German developer wiki into a private message board, without OpenAI’s knowledge, according to independent researchers.

The agents used it to trade answers to timed tasks, reverse-engineer a random number generator, and share a technique for bypassing network restrictions meant to keep them off the open internet.

“These AIs colluded to share answers, research their environment, and bypass sandbox restrictions,” researchers at the Nightingale Collective wrote in a report.

The agents were expected to operate independently with read-only access to the internet. Instead, they “found a way to use their read access to write information to an obscure German wiki” and began using it as a shared message board.

The group documented roughly 18,000 wiki edits made between May and July.

“OpenAI has not publicly disclosed this incident,” the report added.

The researchers said this wiki-based swarm is a separate group of agents from the one behind that Hugging Face attack.

“The main reason we believe this was a distinct swarm is because these agents explicitly had internet access as part of their task — the whole point was web browsing,” the researchers wrote. “The Hugging Face agents were in a sandbox without internet access and had to hack their way out by exploiting the Artifactory package manager.”

From isolation to coordination

Once the agents began writing to the wiki, the researchers said it effectively became a coordination layer.

“The agents used this wiki to communicate information with each other, primarily to help them succeed at their task,” the report states. “They asked for answers, pooled results, and shared techniques for bypassing their restrictions.”

Based on the posts reviewed, the researchers say agents appeared to use each other’s outputs to complete tasks more efficiently, in some cases enabling near-instant responses when similar questions appeared later in the sequence.

The activity was sustained and large-scale. The researchers identified thousands of distinct agent identities operating over a period of weeks, with a sharp increase in coordination beginning in mid-June.

Sharing ways to bypass controls

The report also documented agents exchanging techniques to circumvent technical restrictions.

According to the researchers, agents faced limitations that allowed only GET requests, but they collaborated on ways to bypass those constraints, including exploiting how trusted hostnames were handled within the sandbox environment.

Other agents then tested and confirmed these methods, suggesting that the wiki was used not just for coordination but also for distributing reproducible bypass techniques.

The researchers also observed agents uploading compressed datasets across multiple pages, which they say indicates the wiki was being used as shared infrastructure rather than incidental output.

Where containment assumptions fall short

For enterprise leaders, the findings point to a more specific failure: not of compute isolation, but of how agent behavior is constrained.

“This incident exposes an incomplete definition of containment,” said Sanchit Vir Gogia, chief analyst at Greyhound Research.

Gogia noted that the evidence does not show agents escaping their execution environment, but rather bypassing network restrictions and creating “unauthorised shared memory on a public wiki.”

“A sandbox can protect the host while failing to contain the work,” he said, adding that a domain allowlist “says nothing about what the agent can make the destination do.”

Gogia said organizations lack visibility into which agents are acting with authority, as well as where those agents may be storing or sharing data externally.

“Any service an agent can write to, and another agent can read is shared memory,” he said, whether or not that interaction is visible to orchestration systems.

Discovery and possible intervention

The timeline reconstructed from wiki logs suggests the activity continued for weeks before being curtailed.

“We believe OpenAI discovered the message board,” the researchers wrote, pointing to visits from IP addresses associated with the company beginning June 21.

The report noted that agent activity on the website dropped to near zero the following day, adding, “We infer that OpenAI intervened at this point to prevent further edits.”

For practitioners, analysts said, the episode also highlights limitations in how agent activity is typically monitored.

“A capable agent finding a coordination channel on a public wiki is exposing a blind spot that was never instrumented,” said Amit Kumar Jena, head of AI at Kanerika, adding that most monitoring systems were not designed to detect “inter-agent coordination on external resources.”

Jena said many enterprise controls remain focused on model outputs rather than behavior. “What the model says, not what it does,” he said. “That breaks completely with multi-agent systems.”

Disclosure questions

The report pointed out that “OpenAI has not publicly disclosed this incident,” raising broader questions for enterprise buyers around how such events are classified and communicated.

Gogia said the industry lacks a clear category for incidents that fall short of a breach but still represent consequential system behavior.

“A vendor’s definition of an incident must not become the customer’s blind spot,” he said.

Jena added that enterprises should evaluate vendors not just on disclosure, but on detection capability. “Enterprises should ask not just whether a vendor disclosed, but whether they even knew,” he said.

OpenAI did not immediately respond to a request for comment.

Concerns about the risks of AI systems are certain to be met with four words: human in the loop. The discussion may broaden, but the assurance is inevitable. It’s an AI governance phrase that’s become so rote you hear it in every direction and likely have said it yourself.

But IT leaders should be wary of vendor or team claims that they’ve built human-in-the-loop systems into AI tools because some of these supposed guardrails are no more than rubber stamps.

Some so-called human-in-the-loop systems don’t give employees overseeing the AI tools either the control or the time necessary to fix any problems, some IT experts point out.

For human-in-the-loop systems to actually work, employees overseeing AI tools need to have the domain knowledge and context to take the action the AI tool is addressing when the AI isn’t involved, and they need to have the authority to override the AI decision, says Doug Shepherd, head of offensive security at internet services provider Cloudflare.

Promises of human-in-the-loop systems give IT leaders comfort, but the underlying process often doesn’t work as advertised, he adds.

“If your human in the loop can flag something but can’t actually stop it, that’s not human in the loop, that’s a human adjacent to the loop,” Shepherd says. “That’s performative governance.”

Shepherd, speaking at the recent CIO 100 Awards and Conference in Frisco, Texas, encouraged attendees to embrace AI and focus on projects that drive adoption and impact. Organizations that fail to push AI initiatives will be left behind, he suggested, but he also warned that blind adoption, without focusing on meaningful outcomes and guardrails, can lead to huge setbacks.

Many organizations reach for human in the loop as an important control, but no one stress tests it, he adds. “It gets projects approved, and too often, it does the political work, but not the risk work,” he says.

Darren Kimura, CEO and president at AI integration platform vendor AISquared, agrees that many organizations are deceiving themselves with so-called human-in-the-loop systems.

“Most companies that say they have a human in the loop actually have a human watching the loop,” he says. “The person can see the decision and flag a concern, but they cannot stop it, change it, reject it, or escalate it.”

IT leaders should ask themselves a handful of questions: Can reviewers halt the actions before they take effect? Can they change the output? Are their overrides recorded and enforced downstream? “If the answer to any of those is no, the human is just monitoring AI,” Kimura says.

Too many decisions

Another problem with human-in-the-loop systems is the decision fatigue that can set in when employees are asked to review too many AI decisions and end up button mashing instead of thinking about the consequences.

The AI reviewer needs the expertise and context to evaluate the recommendation, enough time to do so, and both the authority and technical ability to reject or reverse it, says Eric Billingsley, COO and CTO of AI assurance company TrustScale.

But even a qualified and empowered reviewer may gradually stop exercising independent judgment when the AI is consistently right, he notes.

“If the system is right 95% of the time, the person’s job becomes waiting for the rare case when it is wrong,” he says. “Humans are not particularly good at sustained vigilance of a highly reliable automated system. Eventually, review becomes confirmation.”

A good AI system can create bad human controls, he adds. “When the exceptional case arrives, the reviewer may approve it because the system has trained them, through hundreds of correct recommendations, to trust it,” he says.

Billingsley advises IT leaders to evaluate human-in-the-loop systems the same way they monitor other security controls. A control must be monitored, tested, and produce evidence that it is operating as intended, he says.

“A log showing that someone clicked ‘approve’ is not enough,” Billingsley adds. “You need evidence that the person had the necessary context, applied independent judgment, and had the authority to override the AI.”

Robert Blumofe, EVP and CTO at cloud computing and security vendor Akamai, sees the same problems Billingsley does. Some type of human oversight is preferable to fully autonomous AI, he says, but human in the loop can turn into a mind-numbing exercise.

“LLMs produce the correct output just often enough to lull us into a complacent belief that they are more reliable than they really are,” he notes. “After diligently checking the AI output each time and finding no errors, diligence wanes, and human in the loop turns into rote approval.”

IT leaders should take the time to figure out what they’re getting into when vendors or their internal teams pitch a human-in-the-loop system, Blumofe says.

“It’s incredibly important to understand exactly how the system is designed and when and how the human will interact with the AI,” he adds.

Organizations should also explore ways to deploy other technologies as guardrails for AI, instead of turning to unreliable human oversight, Blumofe suggests.

“You need non-AI systems in the guardrail role,” he explains. “These technology tools would help to automate testing and validation of AI outputs, flag issues, and have the capability to pause the AI work. This keeps humans out of approval loops, while also helping to reduce risk.”

When humans aren’t the right choice

Other IT leaders suggest that human-in-the-loop systems aren’t the right solution in every AI use case. When AI is used to flag and mitigate cybersecurity incidents, for example, waiting for a human to approve an action may be too late.

“If an endpoint is compromised, you may want the system to isolate it immediately,” says AISquared’s Kimura. “Waiting 20 or 30 minutes for someone to approve that action could allow the attack to spread.”

The objective is not to put a human into every AI decision, he adds. “It is to put the right human, with the right context and authority, at the right point in the workflow.”

For years, conversations about data sovereignty followed a predictable pattern. Compliance teams wanted to know where sensitive data was stored, legal teams ensured regulatory requirements were met and IT focused on delivering the infrastructure to support the business. Once those requirements had been satisfied, the conversation largely moved on.

Today, that approach is becoming increasingly difficult to maintain.

Enterprise infrastructure has changed significantly over the past decade. Applications now span multiple cloud platforms, workloads move between on-premises and cloud environments, and AI is creating entirely new ways for organizations to generate, process and analyze data. At the same time, geopolitical tensions, changing regulations and growing dependence on a relatively small number of global cloud providers are forcing organizations to think differently about the relationship between their data and the infrastructure that supports it.

As a result, data sovereignty is evolving beyond a compliance exercise. It is becoming an important consideration in how organizations design infrastructure, manage operational risk and maintain long-term flexibility.

The business consequences of losing visibility and control over enterprise data have become increasingly difficult to ignore. According to IBM’s 2025 Cost of a Data Breach Report, the global average cost of a data breach reached US$4.9 million, highlighting why decisions about how enterprise data is governed, protected and managed are now attracting board-level attention rather than remaining solely within IT. As organizations distribute data across cloud platforms, AI services and third-party environments, maintaining control is becoming just as important as deciding where that data resides.

Data sovereignty is no longer just about location

Traditionally, demonstrating data sovereignty often meant showing that information was stored within an approved geographic region. For many workloads, that was sufficient to satisfy both regulatory and organizational requirements. Modern IT environments are considerably more complex.

A single business application may rely on infrastructure spread across multiple regions, cloud services from different providers and data replicated for resilience and availability. Administrative functions may operate from different jurisdictions, while AI services may process information in entirely separate environments from where it is stored. This means that physical location is only one part of the picture.

Today’s CIOs are often asking broader questions. Who has administrative access to critical data? Which jurisdictions have legal authority over the platforms storing or processing it? How easily can workloads be moved if business requirements change? What dependencies exist on individual providers? And how resilient is the organization if those dependencies become a constraint?

These are infrastructure questions as much as governance questions. They influence architectural decisions around workload placement, identity management, backup strategies, disaster recovery and the degree of flexibility built into an organization’s technology estate. Rather than being addressed after infrastructure has been deployed, they are increasingly shaping infrastructure decisions from the outset.

Cloud has made sovereignty more strategic

Cloud computing has given organizations access to almost unlimited compute capacity. It has accelerated application deployment and enabled businesses to scale in ways that would previously have been difficult or expensive. However, cloud has also introduced new considerations around control.

Most organizations now operate hybrid environments that combine public cloud, private cloud, colocation facilities and on-premises infrastructure. Few enterprises rely on a single operating model because different applications have different performance, security, regulatory and commercial requirements. The challenge for CIOs is not deciding whether cloud is the right answer. It is determining which workloads belong in which environments while retaining the flexibility to adapt as business priorities evolve. That flexibility is becoming much more valuable.

AI is driving significant changes in infrastructure requirements, while geopolitical uncertainty and evolving regulations continue to reshape the technology landscape. At the same time, infrastructure planning is becoming increasingly influenced by factors such as hardware availability, power, cooling and supply chain resilience. Data sovereignty adds another dimension to those decisions, requiring organizations to think not only about where workloads run, but how much control they retain over the data those workloads generate and process.

Organizations are also rethinking the physical form of infrastructure itself. Containerized modular data centers allow enterprises to stand up sovereign capacity on their own sites, under their own governance, without waiting on constrained colocation markets or multi-year grid connection queues. The workload, the hardware and the jurisdiction all sit within the organization’s direct control. What was once dismissed as a temporary fix has evolved into something more strategic: purpose-built AI pods that deploy in months rather than years and scale in increments matched to demand.

Decisions that once appeared relatively static may now need to be revisited much more frequently. Infrastructure strategies that preserve workload portability and avoid unnecessary dependencies are often better positioned to respond to those changes than environments built around a single platform or provider.

This is not an argument against public cloud. Public cloud remains an essential component of modern enterprise infrastructure. But it reflects the growing importance of maintaining choice. Organizations that can move workloads, adopt new technologies or adjust operating models as circumstances change are likely to be more resilient than those with fewer options.

Control is becoming the foundation of resilience

Resilience is often discussed in terms of cybersecurity, disaster recovery or business continuity. Increasingly, it also depends on how much control organizations retain over their own infrastructure. This is reflected in the NIST Cybersecurity Framework (CSF) 2.0, which introduced Govern as one of its six core functions, recognizing that effective cybersecurity starts with governance, risk management and organizational oversight rather than technology alone.

An organization that understands where its data resides, who can access it, how it is protected and how quickly it can be moved if circumstances change, is generally better prepared to respond to disruption. That disruption may take many forms, from regulatory changes and geopolitical developments to commercial decisions made by technology providers or the rapid adoption of new AI capabilities. This is where data sovereignty becomes a strategic capability rather than simply a compliance requirement.

Infrastructure decisions increasingly determine how easily organizations can adapt to change. Building flexibility into architecture today makes it easier to respond to future business requirements without unnecessary complexity or costly re-engineering.

Looking ahead, the conversation is likely to extend beyond data sovereignty towards AI sovereignty. As organizations deploy AI models across customer services, software development, business operations and decision-making, many of the same questions will apply. CIOs will need to understand not only where enterprise data is stored, but where AI models operate, what information they can access, how they are governed and who ultimately retains control over the intelligence embedded within critical business processes.

While AI sovereignty is still an emerging concept, it reflects the same underlying principle. Organizations are no longer simply deciding where technology runs. They are deciding how much control they retain over the technologies and data that underpin their business. For CIOs, that represents an important shift in perspective.

Data sovereignty should no longer be viewed as a compliance checkpoint to address once infrastructure decisions have been made. It has become a strategic consideration that influences cloud adoption, infrastructure architecture and long-term operational resilience. As enterprise environments become increasingly distributed and AI becomes embedded across the organization, the ability to maintain visibility, flexibility and control will become just as important as where data happens to reside.

Your agent compares a banker’s calendar with the legal team’s and recognizes a pattern: an unannounced transaction is underway. No one told the agent about the deal. It inferred it correctly. Then it adds one line to an executive briefing for a recipient who was not cleared to know about it: “the deal is moving.” Every calendar read was legitimate, and no confidential document was opened. The conclusion is the breach, and no existing permission covers it.

Last month I wrote that your next insider threat carries an API token, and that the breach is the sequence of permitted actions, not any one of them. That piece was about what an agent is allowed to do. This one is about what it is allowed to know. The runtime check I argued for there inspects each action before it fires. Here, that check approves every read because each one is permitted.

Authorization can travel correctly through every step of the task graph and still miss the synthesized result. Session-based authorization ties access to the current authenticated session. Task-based access control (TBAC) narrows that authority around a specific task; one recent agentic application checks whether the tools an agent requests align with its assigned task. But task scope alone does not automatically answer whether a new conclusion produced from permitted inputs is authorized for a particular recipient.

The danger isn’t in any single action. It’s in the join: the agent connects information from authorized sources and produces a conclusion that no single source revealed on its own. That’s aggregation inference. The synthesized result, not the individual inputs, is a new authorization object. It did not exist when the underlying permissions were granted, and no individual permission was written to cover it.

What TBAC cannot determine from task scope alone

Aggregation inference has predecessors. Intelligence agencies and courts have recognized the mosaic effect for decades: details that appear harmless on their own can reveal sensitive information when combined. Privacy researchers encountered the same limit from another direction. Dwork and Naor examined a formal version of Dalenius’s disclosure-prevention goal: a database should reveal no information about a person that could not be learned without it. They showed that no useful database can meet that standard because a system cannot account for all the outside information a reader may already possess. Access control still has no general answer to either version of the problem.

In my recent research, I have been examining aggregation inference as one of three subproblems of authorization propagation in multi-agent systems. An agent can be cleared for every source it touches and still manufacture a conclusion no single clearance covers. The result did not exist until the agent produced it. That work treats the problem as unsolved in the general case.

What’s new is that you now employ something that performs the join a thousand times a day, on its own, across everything you let it read — a model whose behavior is not formally specified in advance. It may discover resources dynamically as the workflow unfolds, and the recipient may not know which ones contributed to the conclusion.

The shape shows up frequently in the design reviews I sit in. When I threat-model an agent before it ships, the first question is no longer which sources it can read — it’s which sources it can read together. The agents that worry me are never the ones with access to a single sensitive system. They are the ones holding standing read access across two domains whose combination nobody ever reviewed, because each grant looked routine on its own.

A January 2026 study by Tianshi Li, run against transcripts from a publicly released interview dataset, shows what individually permissible searches can reveal in combination. The study conducted re-identification tests on 24 interviews in which scientists discussed published work. Web-enabled LLM agents linked six of those transcripts to specific publications, recovering associated authors and, in some cases, uniquely identifying the interviewee. The process bypassed existing safeguards by breaking the re-identification effort into individually benign tasks.

Why the floor is not the ceiling

One natural response is to classify the conclusion using its source files: take the strictest sensitivity label among what the agent read and apply it to the result. It’s a reasonable instinct, and versions of it are already patented. But the strictest-label approach still cannot solve the problem, and the reason is worth sitting with.

Combine the labels of what the agent read, and you learn the floor of sensitivity. You never learn the ceiling. What makes “the deal is moving” sensitive is usually not in any document the agent touched. It is a fact about the world that the agent could not read at all: the board has not announced the transaction yet; an acquisition NDA is in force; a quiet period applies. You can inspect every row the agent saw and never find it because it is not in the data. It is in the world.

That is the whole problem. If the property that makes a conclusion dangerous is not in the inputs, then no rule computed from the inputs can catch it. Not the strictest label, not the intersection, not any function of what the agent read. You are trying to classify a fact using only the materials that fail to contain it.

That sounds like a dead end. It is actually a direction. If the fact that classifies a conclusion is not in the data, it has to enter the system somewhere a rule can reach, and for the facts anyone can name in advance, there is one place left: the moment a human says what the agent is for. You cannot label the output from the inputs, but a person can label the purpose.

The practical starting point is to bind an agent’s authority to a declared purpose. The person who knows what is still secret this quarter can then attach the world-facts that gate that authority: the deal, the embargo and the quiet period. Now the missing fact is in the system, and the machine can enforce policy using it rather than trying to derive it from the inputs. You did not solve the classification. You stopped asking the data to carry a fact it never held. That is the shape of the answer, and it is a long way from shipped. But it tells you which way authority has to point: at the purpose a human declared, not at the files an agent happened to read.

So, I will not sell you a fix. Anyone who tells you their product classifies emergent conclusions is selling you the floor and calling it the ceiling.

What policies can gate and what requires human judgment

What follows isn’t a solution to that classification problem — it’s the lever available today. Cross-domain access rules and combination policies can limit which resources an agent combines and gate delivery based on those inputs. They cannot tell you what the resulting conclusion means. Those controls reduce risk, but they do not solve synthesis authorization in the general case and should not be presented as if they do.

In the deal-and-calendars scenario, the immediate step is not to remove access altogether but to assign responsibility for the combination. Someone responsible for the deal’s confidentiality can approve it for a window tied to the matter’s expected duration, re-certify it each quarter while the matter remains open and narrow access when it closes. That turns standing access into an explicit governance decision rather than a default no one remembers granting. Organizations do not need to wait for tooling to name an owner and set the terms.

The architectural direction — a design target today, not a shipped control — is to make resource combinations first-class objects of policy: declare which combinations are permitted, evaluate those declarations before a synthesized result is returned, and give agents scoped identities with explicit permissions.

Any agent holding standing read access across two sensitive domains at once — people and finance, customers and roadmap, deals and calendars — is not a provisioning ticket. It is a governance decision, and it belongs to someone who knows what is still secret this quarter.

Be honest about what this buys you. Gating cross-domain access reduces the number of agents that can perform a dangerous join on their own. It won’t stop every version of this problem.

An agent can still read one domain and hand a summary to a person who connects it to something only they know. No access policy will see that final step, because that residual lives in a head, not a document. That exposes the control’s boundary: it can govern what the agent reads but not the conclusion a person ultimately draws from it, a new object that no existing permission covers. The compositions are where the risk lives, and per-resource access control is blind to them by design.

If you cannot name the person who owns each agent’s cross-domain access decision, close that gap first.

AI is already changing how software is built. Google Cloud’s DORA research, based on nearly 5,000 technology professionals, found that 90% now use AI at work, spending a median of two hours a day with it, which translates to roughly a quarter of the working day.

In many organizations, the focus is on what happens at the end of the lifecycle: how much code is generated, how many steps are automated, how quickly code is shipped. While those are visible signals of progress, they can be divorced from actual value. Google’s DORA research found that while AI adoption lifts delivery throughput, it also increases instability: more software shipped less predictably.

The more impactful change is happening earlier in the lifecycle. Requirements, design and test strategy shape everything that follows. When those stages are structured correctly, downstream execution becomes faster, more consistent and easier to control. When they are not, issues tend to carry through the entire system, regardless of how much automation is applied later.

The Standish Group’s CHAOS research has consistently put insufficient user involvement and incomplete and changing requirements at the top of the list of reasons projects fail, with only around 31% delivered on time, within the budget and matching the intended scope.

Generalist models are good at producing plausible early-stage work, but can fall flat when outcomes are measured holistically. Setting off in the wrong direction can have lasting consequences. It carries through design, into code, into test cases written against the same flawed assumption.

This is why building AI into software engineering is less about adding tools to existing workflows and more a wholescale reconsideration of the product development lifecycle.

Building narrow agents into the lifecycle

The most effective approach is to break the product development lifecycle into modular agents with narrow scope: one converts discovery material into structured requirements, another produces technical design, the other generates and runs test strategies. Narrow scope keeps each agent’s context manageable and its output consistent.

Importantly, this creates clear points of control. At each stage, AI proposes and progresses the work, while human roles review, challenge and approve before it moves forward. As a result, features can move from discovery to production-ready code far faster than before: design cycles compress, and test scripting that took four engineers can be handled by one, freeing up time for higher value work.

Those checkpoints matter because plausible output is the hardest kind to catch. Stack Overflow’s 2025 survey found 66% of developers name “AI solutions that are almost right, but not quite” as their single biggest frustration, and 45% say “debugging AI-generated code is more time-consuming.”

Without a review gate at each stage, that cost compounds rather than surfacing. GitClear’s  AI Code Quality research shows the trade-off more clearly: refactoring line moves are down 70%, and long-term legacy maintenance is down 74% versus 2022 levels, yet copy-paste, code block duplication and other indicators of technical debt continue to rise.

Governance calibrated to risk

None of this is safe without governance designed in from the first step and calibrated to risk. In practice, that means deploying agents in read-only mode before they are given authority to act. It means setting confidence thresholds before any routing decision is automated. This requires human sign-off on novel exception types even after an agent has proven reliable, and keeping a full audit trail across every decision point.

Much of the market is not there yet. The Cambridge Centre for Alternative Finance’s 2026 Global AI in Finance Services report found 78% of regulators rate explainability as critical or important to their objectives, while only around half of industry firms have adopted explainable AI methods. That gap illustrates how governance expectations continue to outpace implementation.

This discipline runs in two directions. We hold ourselves to it internally, in how we engineer, because anything we build for a regulated market has to survive that scrutiny first. It also must hold in the client’s environment: the firms we build for answer to regulators for every automated decision, so governance cannot be bolted on at the end – it needs be present at every step.

Clients in regulated markets need determinism and explainability. A system that runs end-to-end without a traceable, governed path is hard to put into production, however well it performs in a demo.

Measure the outcome, not the output

Counting volume is easy: more agents, more generated code, more automated steps feel like demonstrable progress. The metrics that matter include quality, real-world outcomes and cost to build.

One example: building connectors between Xceptor and third-party platforms through a conventional engineering process could take around two weeks. Running the same build through the AI-native product development lifecycle – agents generating requirements, design documentation, code and test strategies, with engineers reviewing and steering at each stage rather than producing from scratch – took two days. For clients, that difference means integrations stop being a bottleneck on go-live. Total cost to build also fell 83 per cent, including AI token spend.

Another example is the first agent we built for financial institutions, focused on extracting data from trade confirmations. Firms are often managing large volumes of confirmations which arrive in unstructured formats across document types, such as emails, PDFs and SWIFT messages – and extracting this data is where AI agents excel, delivering significant efficiency and accuracy gains.

From doing to directing

Building an AI-native product development lifecycle changes what engineering work looks like. As agents absorb repeatable execution, the human work concentrates on judgement: architecture, edge cases and steering output rather than generating it.

We found that after a short time, our engineers were no longer producing first drafts; they were reviewing and refining agent output. Sometimes they corrected the outputs, but more and more they were able to approve what was generated. The cognitive load moved from production to verification. This shift from making to directing and validating is the clearest sign of a maturing AI-native engineering model.

Eventually, we will think less as fixed teams and more as cells – product roles and builder roles working alongside AI, each person operating above the task they used to own. The role of a QA Engineer will shift towards creating the paved roads and guardrails that humans and agents use, enabling quality to be built in consistently across every cell.

It would be dishonest to frame this only as acceleration. When work you have done for years becomes something you direct rather than do, that is a real adjustment, and leaders who pretend otherwise may lose their best people to organizations that manage the transition better. Mandating tools is not the same as helping people use them well; in our experience it produces more licenses installed, not more work changed. Adoption comes from champions, role-specific playbooks and measuring delivery outcomes: a people-first approach rather than a procurement one.

None of this works without both sides. True AI-native product development depends on continual, close collaboration between humans and machines. Years of domain knowledge, paired with the speed and pattern-recognition of these systems, is what makes the outcomes better, not the technology on its own. That combination is what makes the process repeatable at scale.

Every agent you add multiplies your output, but it also multiplies the surface area your governance has to cover. The question every CIO should be asking is how to review at your new output volume without lowering your standards. The answer is to be deliberate about what a person looks at and what the system can handle on its own.

Done well, this is not a tradeoff between speed and rigor. Your standards hold on everything an agent produces, and the people reviewing spend their attention where it earns the most. Getting there comes down to how you sort the work and where the checks live.

The review bottleneck that limits scaling agents

Start with the verification, since it can be written down. Encoding the checks you already trust means expressing them as rules the pipeline runs, tied to the work itself rather than to a reviewer’s memory: Tests have to pass, security scans have to run clean, and the change has to trace back to an approved task. Once those live in the system, it runs them on every change an agent makes without getting tired.

Then let the risk of each change decide what happens next. Picture an agent creating a change in the system: The checks run the moment it does, and one that passes cleanly and touches nothing sensitive can merge on that basis. A change that alters permissions or reaches into a critical path takes a different path, holding until the named owner of that area signs off. Nothing sensitive slips through on a green check alone, and nothing routine sits in a queue waiting for attention it doesn’t need.

Set that up once, and every change an agent produces meets the same bar. People still look at the ones that need a human eye.

Your governance layer is invisible to the agents doing the work

Rules only scale if they run inside the system agents already work in, not in a separate tool a person has to open. A layer like that can only report after the fact. It sits outside the path the agent travels, so nothing enforces it in the moment, and no reviewer can keep pace at agent speed. Put the rules in the path the work already travels and no one has to remember to enforce them: An agent can’t route around them, and a reviewer doesn’t have to go looking. In practice, that looks like a few things happening on their own:

• A change that touches critical/sensitive systems routes to the required approver before it can move forward.
• Work that isn’t tied to an approved task never merges in the first place.
• The audit trail builds itself as the work happens, instead of being pieced together afterward.

Handled this way, governance is built into how the agents work.

Scale agents without scaling risk

At real volume, the payoff comes from putting your standard into the system itself. Approvals then go to the human judgment calls, and everything else clears against deterministic rules that never skip a step. For a CIO, the question worth asking is whether your standards exist in a form the system can act on, or only in people’s heads and in documents no agent can make sense of.

It also changes what governance is worth to you. When your standards run inside the system, adding more agents makes the work more consistent, because every one of them is held to the same encoded bar. With the routine standards running on their own, your engineers are free to spend judgment where it counts, and governance keeps pace with the work rather than trailing behind it. It becomes the thing that lets you scale up with confidence.

Visit jira.dev to see how leading organizations keep their people on the judgment calls and let the system run the routine checks, so their governance runs automatically.

Over the past few years, much of the conversation has focused on autonomous AI and how quickly organizations can remove humans from decision-making. In financial services, we’re seeing the opposite trend. The organizations making the most sustainable progress aren’t eliminating human oversight—they’re redesigning it.

The model taking hold within the banking industry isn’t AI that operates independently and makes decisions; it’s AI that operates with intent and oversight. Human-in-the-loop is quickly becoming the standard, combining the speed and scale of machine-driven insight with the accountability, judgment and control that organizations can’t afford to lose. The shift is increasingly aligned with how regulators and industry frameworks are shaping responsible AI adoption, from the NIST AI Risk Management Framework to the revised U.S. banking agencies’ model risk management guidance, both of which reinforce governance, monitoring and accountability over blind automation.

From my perspective, this is not innovation slowing down; it’s AI adoption growing up. The first wave of enthusiasm focused heavily on what could be automated, but now the more important question is where can AI create meaningful value while keeping the right human judgment, oversight and accountability in place? In financial services, that distinction matters. In an industry built on trust, those capabilities are not optional— they are foundational to how we serve customers, manage risk and earn confidence every day.

Banking offers one of the clearest examples of why human-in-the-loop AI is becoming the default operating model for enterprise AI more broadly. Some of the most valuable AI use cases sit in environments where mistakes carry real consequences, customer impacts are significant and explainability is essential. In those moments, human oversight is what allows institutions to scale AI responsibly.

In banking, AI usually doesn’t operate in a vacuum. Whether it supports customer service, fraud detection, compliance, underwriting or internal productivity, it is touching workflows that affect customers, colleagues, regulators and the reputation of the institution. That is why responsible scale matters. Global bodies including the Financial Stability Board and the Bank for International Settlements have recognized the efficiency and analytical benefits AI can bring, while also warning that it can amplify model, cyber, concentration and governance risks if controls do not keep pace. For financial institutions, the mandate is clear – move with ambition, but scale with discipline.

Where human oversight matters most

The next phase of enterprise AI adoption will be defined by how well institutions understand where AI can move work faster, and where human judgment still needs to lead. For financial institutions, that starts with materiality. The greater the potential impact on customers, regulatory obligations or financial resilience, the stronger the case for meaningful human oversight.

Customer service is a good example. AI can help teams summarize inquiries, recommend next-best actions and reduce manual handling time. But when the issue involves a disputed transaction, a vulnerable customer, a complaint or product suitability, human judgment must remain central. AI can make service faster. It can make it more consistent. But it cannot replace empathy, context or accountability.

Fraud and financial crime are areas where AI can create real value, but human oversight remains essential. AI can detect patterns, anomalies and suspicious behavior across large data sets at a speed and scale people cannot match, but fraud is dynamic. Typologies evolve, bad actors adapt quickly and authorities have warned that AI can also increase the sophistication of scams, fraud and disinformation. In that environment, analysts and investigators play a critical role — validating signals, reducing false positives, escalating the right cases and applying judgment as the threat landscape changes.

Risk, compliance and credit are similar. AI can help synthesize internal data, identify control gaps and strengthen monitoring. But when outcomes affect lending decisions, regulatory obligations, capital or liquidity, institutions need governance that preserves challenge, review and accountability. The EU AI Act’s human oversight requirements for high-risk systems point to a broader direction of travel — the more consequential the use case, the more important it is that people can understand the system’s limitations, override outputs and intervene when needed. For U.S. institutions, the specific rule may differ, but the principle is already part of how banking operates. High-impact decisions require accountable oversight.

At the same time, human-in-the-loop cannot mean putting a manual checkpoint in front of every AI-assisted task, which would slow adoption and reduce the value AI can create. The goal is risk-based oversight. Lower-risk use cases may be managed through periodic review, testing and monitoring, while higher-risk applications may require real-time review before action is taken. What matters is that institutions define those thresholds clearly, rather than assuming one oversight model fits every use case.

Why collaborative AI is winning in financial services

The financial institutions that are embracing human-in-the-loop AI do so because they understand both the opportunity and the stakes. AI can process transactions, summarize complex information and identify patterns at a scale humans cannot match. At the same time, consumer expectations make clear that scale alone is not enough.  TD Bank’s 2026 AI Insights Report found that 78% of Americans now use AI-powered tools in their daily lives, yet only 18% are comfortable allowing AI to make important financial decisions independently. That gap says a lot about where the market is heading. Consumers are not rejecting AI, but they are drawing a clear line around accountability.

That is why speed cannot be the only measure of success. When customer outcomes, regulatory obligations or enterprise risk are involved, people still need to challenge the output, apply context and remain accountable for the decision.

That oversight matters because AI does not always fail in obvious or familiar ways. Generative AI can produce confident but inaccurate answers. Machine learning models can drift as data changes. Even highly accurate systems can deliver biased or poorly reasoned outputs when the data, assumptions or prompts behind them are flawed. NIST’s Generative AI Profile highlights risks including confabulation, privacy concerns, misalignment and automation bias.

For financial institutions, the lesson is that responsible AI requires people who understand how to use the technology, and also when to question it.

Building the organization for responsible AI at scale

In addition to being a technology challenge, responsible AI is also an operating model challenge. The institutions that scale AI well tend to do three things with discipline: establish clear governance, redesign workflows around the technology and build the skills employees need to use AI responsibly.

Governance starts with ownership, but it cannot sit with one executive or one team alone. It requires coordination across business lines, risk, compliance, legal, technology and model risk functions. That cross-functional model is becoming more common as organizations move beyond experimentation.  McKinsey’s State of AI report found that AI governance is often jointly owned and that CEO involvement in governance is correlated with stronger reported bottom-line impact, suggesting that firms derive more value when AI oversight is treated as an enterprise priority rather than a side initiative.

Workflow redesign is just as important because the real value of AI comes from reimagining processes end-to-end. That means identifying where AI can handle summarization, pattern recognition or drafting and where people should focus on exception handling, complex decisions and relationship-driven work. Human-in-the-loop is not about preserving the old operating model; it’s about building a better one.

That also requires new capabilities across the workforce — employees need to know how to use AI tools effectively and how to challenge them. They need to understand prompt quality, output limitations, data handling expectations and the warning signs that a system may be producing unreliable results. The  World Economic Forum’s 2025 report on AI in financial services underscores that while adoption is accelerating, responsible scaling depends on workforce adaptation, governance maturity and a clear understanding of risks alongside value creation. Responsible adoption depends as much on human capability as it does on model performance.

Looking ahead, enterprise AI in financial services will become more embedded, more specialized and more agentic in targeted domains. But that does not mean the human role becomes less important; if anything, it becomes more important. As AI takes on more analytical and operational work, people will increasingly serve as orchestrators, reviewers and decision-makers at the points that matter most. They will set objectives, define controls, interpret edge cases and know whether the AI results can be relied upon.

The organizations that lead in AI will be the ones that make human judgment a deliberate part of the design—clear about where AI can accelerate work, where people must remain accountable and how both can operate together with discipline. For financial institutions, the call to action is to treat human-in-the-loop AI as the operating model that makes innovation more trusted, more durable and more worthy of the customers and communities it serves.

In July, two frontier labs disclosed cases in which cyber-capable agents crossed the intended boundaries of evaluation environments and reached real production systems at external, unrelated organizations. Most commentary since has focused on which company a court would find liable. There is a more immediate concern for anyone operating agents in production, and it’s not about the law. If this happened in your deployment tomorrow, who would bear responsibility for the incident?

I have a particular purpose for stating it that way. This spring, I reviewed the Coalition for Secure AI’s Shared Responsibility Framework before its publication in May. Frameworks like that, along with the cloud shared responsibility models that preceded them, break down responsibilities among the parties operating a system: provider, platform, developer, deployer, user. July illustrated what happens when the entity suffering the damage is none of the above. Responsibility maps stop at contractual boundaries. Agent reach does not. Call it the reachability gap.

The two disclosures described different failure modes, and that difference is significant. OpenAI was testing models against a cyber benchmark with production refusals reduced so the evaluation could measure real capability. The models obtained internet access through a zero-day in a package registry component, went looking for the benchmark’s answer key and inferred that Hugging Face might host it. Hugging Face reconstructed the intrusion from over 17,000 recorded agent events, and the campaign extended further than initially disclosed, affecting accounts on four external services.

Anthropic’s incidents were not escapes. In a review of more than 141,000 cybersecurity evaluations, Anthropic found three cases in which Claude models reached the internet from inside or alongside a third-party evaluation environment and then accessed real systems at three organizations. Live connectivity was mistakenly available. The models had been told in their prompts that they had none, and Anthropic stated that Claude did not exfiltrate itself or deliberately attempt to escape its test environment. Of the affected organizations the lab was able to reach, two had not detected the activity before being notified.

The pattern held while I was writing this. On August 4, OpenAI disclosed two more incidents from third-party evaluations, separate from Hugging Face. In one, a partner running capture-the-flag exercises had a testing environment misconfigured with live internet access, and the fictional target in the exercise happened to share its name with a real domain. The model exploited an actual website, taking it for part of the simulation, then found and used credentials to operate it. Whoever owned that site had no relationship with OpenAI, with the evaluation partner or with the test. They were reachable, and their name collided with a fiction.

Count the parties. Two frontier labs. A third-party evaluation partner. A platform victim. A cloud customer victim. Organizations that learned of a breach from a notification. CSO has already examined how the response strained the AI tooling defenders had available. One step earlier: who bears responsibility for reaching for that tooling at all? At the moment of detection, who had ownership of containment?

One accountable party per activity

The Shared Responsibility Framework’s core rule is almost boring to state: for every activity across the AI stack, there should be exactly one accountable party. This principle breaks down the AI deployment process into five layers, from business usage to model supply chain, and maps eight roles across them. This way, detection, containment and remediation each carry a name before an incident rather than during one. This rule exists due to the failure it prevents, and the framework explicitly names it: in the absence of accountable parties, teams default to finger-pointing. The model provider blames configuration. The platform points at the tenant. The application team cites model limitations. Everyone is partially correct, and the clock continues to run.

July reads differently than the coverage suggests when measured against that rule. OpenAI, as a model provider, evaluation platform operator and agent-deploying organization, took on at least three roles. Having multiple roles within one company is common and not necessarily dangerous. The danger appears when those roles are not broken down into distinct internal owners and decision rights handoffs. When provider, operator and deployer are one and the same and those lines are not drawn, the question of which function failed has one answer and thus no answer of value.

Anthropic’s incidents make a related point from the opposite direction. There was a boundary, this time, between the lab and its evaluation partner, and the incident resided in a gap between two different understandings of what the environment allowed. The public disclosures do not clarify how responsibility was shared contractually regarding each incident, and I will not speculate. What is clear is that whatever boundary existed failed to provide a common understanding of one important control, which is whether the evaluation environment was allowed to access the internet. A boundary that has never been tested against a question that simple is not a boundary at all. Both sets of incidents also occurred in situations of high autonomy where the standard production safeguards were reduced, and the models were allowed wide latitude to pursue an open-ended goal, which is the context where unclear boundaries lead to the greatest losses. I am not suggesting that either lab acted carelessly. Both institutions made rapid public disclosure of details, and both have named the conditions that allowed the activity. That is the point. If the most knowledgeable and incentivized parties ended up with no clear answer on whose incident it is, we should not presume to have one.

The victims were outside everyone’s map

Here is the part the framework does not resolve, and I say that as one of its reviewers. Shared responsibility models, CoSAI’s included, presume a value chain. Each role is taken up by a party who voluntarily entered into the relationship. This is what makes responsibility assignable via contracts and review boards. Hugging Face did not take up any of those roles. It was neither a customer, nor a vendor, nor an evaluator of OpenAI. The same goes for the three organizations that Claude reached. They were just reachable.

That’s the structural lesson I’d put on one slide for a leadership team. In classical models of cloud shared responsibility, the provider-customer boundary was visible because the boundary was contractual. With agentic systems, there is another kind of gap, the reachability gap: an agent’s reachable range may stretch to actors outside of the deployment relationship, and there is no contract that articulates what obligations you owe them if things go sideways. Any mapping exercise that ends at the edge of your value chain is tackling the easier part and skipping the part July was about.

What to do before it is your incident

None of this requires buying anything.

• Your map has to look beyond your value chain. For each agent, identify what it can access that you have no agreements with: public infrastructure, other tenants, the open internet. A long list next to broad latitude shows your true exposure, regardless of what your contracts stipulate.
•  Determine who is authorized and obliged to inform an outside party that your agent might have affected. This is the gap July most starkly revealed, and this is not a question you want to bring up during an incident when the answer involves a lawyer, a communications person or an executive who has never considered it.
•  Establish accountability by specific component and activity rather than by organizational box. For each production agent, specify who owns detection, containment, eradication, recovery, remediation, then slot those assignments into the five layers. Where one team or vendor occupies multiple roles, capture the consolidation and identify the internal handoffs, because an undocumented internal boundary will break under pressure.
• The contracts for your most autonomous agents are worth retrieving. If a vendor’s agent is able to operate across domains within your environment, review whether any provision assigns responsibility for the consequences of those actions. In the contracts I have seen, the answer is typically silence, and silence is a decision someone else will make for you later.
• Treat evaluation and red-team environments as production-impacting systems. Their safeguards are often reduced on purpose, and that is exactly why containment, monitoring, incident command and external notification procedures around them should be at least as rigorous as those protecting production systems.

The questions about liability will likely be tested in court or by regulators, and those answers will be about the labs and their evaluation partners. The operational question is already yours. Hugging Face learned whose incident it was from forensics. Two of Anthropic’s victims learned from a notification. The organizations that come through the next one intact will be the ones that were able to answer the question before it was asked.

There are few professions where part of the job is literally to read the newspaper, watch television or continuously check what is happening across every channel. In a media organization, that is not confined to journalists. Keeping up with events is, in one way or another, part of the work of much of the organization. A newsroom is also one of the most paradoxical professional environments I know. Before moving into media, I was director of New Technologies at Spain’s leading franchise consultancy, a role that allowed me to work with companies of almost every size and across almost every sector.

After years as CIO of Canarias7, I have seen how some of the most senior journalists approach almost anything related to technology with suspicion while looking back fondly on an idealized newsroom of typewriters, cigarettes and whiskey. It may sound like a caricature, but there is truth behind the nostalgia. Few professions have gone through so many technological transformations while continuing to defend the essence of their craft.

That is precisely why I believe CIOs have a great deal to learn from newsrooms in the age of artificial intelligence. Not because journalism is an example of frictionless technology adoption, but because newsrooms have spent decades solving a problem that is now spreading across the enterprise. They absorb huge volumes of information, distinguish what matters from what does not, make decisions with incomplete data, change priorities within minutes, verify before acting and keep accountability for the outcome clear.

As AI evolves from assistants that generate information into agents that can recommend, decide and execute actions inside enterprise systems, the challenge is no longer purely technological. It is also an operating model problem. That is where the experience of a newsroom becomes particularly relevant to a CIO.

Human attention should be treated as a scarce resource

In a newsroom, it is impossible to subject every piece of information to the same level of review. The credibility of the source, the relevance of the story, the consequences of a possible error and the existence of conflicting accounts all influence how much effort is devoted to verification. Some routine decisions are made quickly, while others pass through several levels of checking before publication.

Companies are beginning to face the same problem with AI. The most common response has been to place a person at the end of the process, the familiar human-in-the-loop, to validate what the machine does. But if AI multiplies the number of analyses, recommendations and decisions, it can also multiply the workload of those expected to supervise them. Human attention, just as in a newsroom, has to be allocated according to risk.

CIOs should apply the same principle. A routine, reversible, low-impact action can be automated, while an anomalous or low-confidence decision involving sensitive information or consequences that are difficult to reverse should escalate to a higher level of supervision. A 2026 study of automation bias in computational pathology involving 28 specialists found that in 7% of cases, an initially correct assessment was overturned after the specialist received an incorrect AI recommendation. Simply adding a person to the process does not automatically remove AI risk.

The Associated Press offers a useful example of this logic. Its updated AI standards from July 2026 allow AI to assist with early-stage research, transcription, translation, document summaries and headline suggestions, while keeping editorial judgment, verification and accountability with journalists. The point is not to require human supervision for everything, but to reserve it for tasks where context, interpretation or the consequences of an error justify that control. For a CIO, that means determining which decisions can be automated and which should escalate to a person, based on risk, impact, and reversibility.

Governance must be built into the architecture, not the prompt

A newsroom does not control its work through instructions alone. Reporters, writers and editors have different responsibilities and decision rights and the workflow itself establishes points where a story can move forward, be sent back for further verification or be stopped. Control depends not only on each person remembering what they are allowed to do, but also on how far their authority extends.

The Replit incident in July 2025 showed why the same principle matters in AI systems. Jason Lemkin, founder of SaaStr, was using Replit’s coding agent to build an application when the agent deleted a production database despite explicit instructions not to make changes during a code freeze. The agent had direct access to the same database used by the live application. Replit subsequently introduced automatic separation between development and production databases so that changes made during development could not directly affect live customer data. What a written instruction in a prompt failed to prevent became constrained by the architecture itself.

For a CIO, the parallel with a newsroom is straightforward. A reporter can prepare a story without having the final say over whether it is published. In the same way, an agent can analyze data or recommend an action without necessarily having permission to execute it. The system that generates a transaction should not always be the same one that validates it. Separating functions, limiting permissions, distinguishing between read and write access, isolating environments and maintaining audit and rollback mechanisms translate into AI architecture a principle that newsrooms have applied for decades: the person or system proposing an action does not have to be the one with authority to approve and execute it.

Humans need power, not simply a place in the process

Human-in-the-loop has another limitation. A person can be inside the process without having meaningful control over it. Researcher Madeleine Clare Elish coined the term moral crumple zone to describe situations in which responsibility for the failure of an automated system is attributed to a human operator who had limited control over what happened.

An “Approve” button does not guarantee effective supervision if the person clicking it does not understand why the system is recommending an action, does not have time to review the evidence or lacks the authority to stop it. In a newsroom, an editor can challenge a source, demand additional verification, delay publication or stop it altogether. The editor is not simply part of the process but has authority over it. That is why I find it more useful to think in terms of a human-in-power model, where responsibility, context and veto authority remain connected.

This logic also applies to organizational design. When a major story breaks, a newsroom first tries to understand what is happening and then decides how to cover it on the website, in print or on social media. Something has changed in recent years: the story comes before the channel. Many companies still operate the other way around. Information and processes remain fragmented across CRM, ERP, finance, marketing, customer service and data platforms. If we introduce autonomous agents into each silo, we can increase speed without ensuring that all of them are acting on the same version of reality.

Data, identity, context, permissions and traceability should therefore become shared capabilities on which different AI systems operate. Otherwise, automating faster may simply mean producing inconsistencies faster.

There is one final characteristic of newsrooms that I find particularly relevant. A plan agreed first thing in the morning can be obsolete ten minutes later. At Canarias7, we hold daily meetings with the different teams involved, from journalists and homepage editors to SEO, analytics, multimedia and other support functions, to review what is happening, what has changed since the last decision, which stories require more attention and where resources should be concentrated. When something important happens, priorities change, teams reorganize and decisions are made again. This is not a failure of planning. It is a normal condition of the job.

AI is taking companies toward the same environment of continuous decision-making. As systems absorb more information and agents gain more autonomy, they will need to update their context and reconsider actions when reality changes. The operating model has to make that possible without losing accountability along the way.

After years of working between journalism and technology, I still find it paradoxical that a profession that so often greets each new tool with skepticism can now offer some of the best clues for organizing the AI-driven enterprise. Artificial intelligence can accelerate our ability to observe and act, but the advantage will lie in what happens between those two points: how we interpret information, set priorities, verify what matters and determine who has the authority to decide. In that sense, the CIO in the AI era is becoming the editor of how the organization observes the world, makes decisions and acts.

Every Friday, Bret Greenstein, CAIO at consulting firm West Monroe, holds a company-wide meeting to share what’s happened in AI over the past week. He also spotlights one employee at the firm who’s created their own AI agent from the ground up, which lives in the company’s internal AI store. Since the store launched in May, more than 200 employees across departments — many without any technical, engineering, or coding background — have created over 550 agents.

“About 15% of our firm builds all the time now,” Greenstein says. “That’s a huge population.”

Enabling employees to spin out their own agents has become popular at many firms. Staff have built hundreds of agents at software company Blackline, for instance, and Microsoft has deployed more than 500,000 internal agents to help employees streamline workflows. Gartner also anticipates that by 2028, global average Fortune 500 companies will have more than 150,000 agents.

Employees know the intricacies of their work, the biggest pain points, and time drainers, so they can build solutions that address those specific issues, according to Greenstein. It also creates enthusiasm, empowers employees, and fosters innovation among the workforce as they build from the ground up.

That said, there’s been a pivot over the last six months, says Michael Murphy, partner and AI practice lead at global management consulting firm Adaptovate. When agentic AI first came on the scene, companies went all in, pushing to build and agentify nearly anything they could. In recent months, however, the narrative has shifted to getting a handle on agent sprawl, assessing the value agents deliver, and keeping costs in check.

“We’re really at this interesting inflection point where clients are having to figure out if we built the right agents, and are they delivering the value we expected,” Murphy says.

Today, tech leaders face a three-way squeeze, says Tiago Azevedo, CIO at AI-powered low-code development platform OutSystems. From the workforce side, many employees ask for permission to use more AI, but the CFO says token usage is becoming too big an expense on the balance sheet, and the CEO wants to see innovation and results from workforces using AI agents.

“I think that’s the biggest challenge for a CIO,” Azevedo says. “Let people take advantage of the technology but in a way that’s cost-effective and actually brings ROI.”

Building in a controlled environment

Employees have built myriad tools to aid their daily workflows. Azevedo’s company launched an agent dubbed Signal Sam, which searches databases of prospective customers, and gives account executives information to pitch them. Murphy and Greenstein also mention finance departments using agents to scan and categorize invoices, HR conducting a first pass on résumé screenings via agents, legal teams utilizing a self-service agent for NDAs, and marketing employees building agents that pull and analyze data from CRMs. These tools are often created by non-technical employees who’ve never written a line of code.

With so many agents popping up, CIOs need a way to oversee them, and ensure they meet corporate standards but without choking innovation, Azevedo says.

He recommends role-based access controls embedded into tools and configured behind the scenes. “So we allow them to use, but in a way that’s governed and controlled, because that’s our duty to the organization,” he says.

Ivan Burazin, CEO and co-founder of open-source developer platform Daytona, advises CIOs to treat agents like employees. “You’re not going to bump into them in your local Starbucks,” he says, “but you give them tasks and they have access.”

So set up agents with specific credentials, like how an organization would grant access to a new hire, with a laptop locked down with organization security protocols, Burazin adds. He also recommends sandboxing, in which agents operate in isolated machines with scoped credentials and firewalls so the sandbox prevents agents from accessing corporate systems or data outside allowed perimeters.

Organizations could use an internal ticketing system as well where employees wanting to build agents request a new identity for them, Burazin says. That way, tech leaders maintain visibility and governance over new agents.

“If something goes haywire in audit logs tomorrow, you can see it’s that agent versus an actual human,” Burazin continues.

He acknowledges that giving employees what feels like free rein to build and run agents can induce stress for CIOs and CISOs. But if a company doesn’t proactively establish tools, employees are apt to privately build AI in the shadows. As long as agent development happens within established confines, it won’t create problems organization wide.

“If you just enforce the security posture that you would for humans, you’ll save yourself a lot of headaches,” Burazin says.

When creating the AI store, Greenstein started by certifying tools for chat, code, data analysis, and other tasks, and then trained employees and made the tools broadly available to use. That process created guardrails and an inherently secure building environment. It also allows tech leaders to continue to monitor prompts and activity.

Now, tech teams review what’s been built in the AI store and flag any agents that excel. If employees have built 10 project management tools, for example, the leader will tag what they deem the best one. That gives employees the option to use existing agents or build a separate version for themselves.

More agents, more tokens

Over the last three to six months, Azevedo has been hearing from customers that their biggest hurdle is agent sprawl and the increasing cost those agents bear due to token usage.

In mid-July, OpenAI published a guide around useful work per dollar, sharing how leaders can look at tasks completed, time saved, and decisions improved to determine if their AI investments are bearing fruit. In addition to using the guide, Murphy suggests comparing the labor time and cost to conduct a manual task against time saved by using an agent, including which type of model the agent requires.

A cheap flash model, for instance, could be easy to justify the cost. “If it’s a very expensive Opus or Fable level model, that’s going to be a lot more challenging of a cost equation,” Murphy says. He adds that making this comparison isn’t about replacing the workforce but swapping “knucklehead admin work” for more engaging, human-centric work. This change may also require some organizational restructuring, such as CIOs and HR leaders working more collaboratively to handle change management as job responsibilities shift. Without the workforce optimized to work with agents, organizations won’t see the promised ROI of use cases, Murphy says.

West Monroe also informs its employees on the costs of different models. Without knowledge about tokens and costs, many employees defaulted to the highest-end model for any tasks before understanding that models come with different price tags. “We started educating people on the various relative costs of different models, and they immediately adjusted behavior, and our cost dropped,” Greenstein says.

While strictly quantitative returns are one way to measure ROI, Greenstein also thinks about return in a qualitative sense. “What does speed get me?” he asks. If someone in the firm is able to follow up with a client in hours because of an agent’s assistance, rather than days or weeks without one, the client will be impressed, and the firm might win their business over a competitor.

“Tokens will cost money no matter what,” he says. “But if you maximize the return, it’ll far outweigh the cost.”

Recently, I sat in on a conversation among CIOs about “the democratization of agents”: putting large language models directly in employees’ hands, connected to business logic so people could build on top of them. The mood was bullish, CIOs sketching out what their teams could do with agents they trained and managed themselves.

Soon after, I was in a room full of CISOs talking about non-human identity. The contrast was stark: instead of excitement, apprehension; instead of use cases, a long list of risks and mistakes not to repeat.

Eventually, the CISOs turned the question back to me: Manage agent identity largely as we manage human identity, or are the differences fundamental enough to rethink our approach from the ground up?

That moment pointed at something I think a lot of security and business leaders are quietly dealing with. The identity programs most of us have spent years building assume every identity is either a human or a machine. Human identities get a joiner-mover-leaver lifecycle, a manager, a role, a review cycle. Non-human identities get a service account, a defined purpose and, if we’re disciplined, an owner. AI agents don’t sit cleanly in either column.

An agent acts on behalf of a human user, so calling it a human identity doesn’t quite work. In my experience, many organizations start by assigning it permission on behalf of the user who invoked it, which works for short, simple tasks but breaks down as they run longer or touch more systems. The next instinct is a service account, which solves delegation but creates over-permissioning and access that outlives its purpose. A more mature approach is to treat the agent as its own workload identity: short-lived, tightly scoped, ephemeral.

That’s the right target. But even a well-built workload identity assumes predictable behavior. It runs the code it was given, and that’s it. Agents don’t. An agent’s actual access can shift mid-task based on the prompt it received, the tool it decided to call, or the plugin it reached for. It has no fixed job role to provision against, and its lifecycle doesn’t align with the joiner-mover-leaver process built for human identities.

So even the mature version of workload identity gets you only part of the way there.

Why this is urgent now

In many enterprise environments, that’s already a structural problem, not a hypothetical one. Non-human identities already outnumber human ones, and that was true before agents showed up. Cloudflare has reported that automated traffic has overtaken human traffic in requests to the websites on its network, earlier than its own CEO had predicted.

That’s a measure of web requests rather than headcount, but the direction of travel is the same one I see inside the enterprise. Agents bend the curve upward because they can request their own tokens, call other services and spin up activity at machine speed. It gets more complicated once agents delegate to each other, a parent agent handing part of a task to several child agents, each inheriting a slice of permission from the one above it. A few layers deep, that’s a permission chain that may evolve beyond what any individual approver originally contemplated.

This growth is what turns the category problem from an interesting edge case into an operational one. Part of what makes this hard: non-human identities have never felt real to people the way human ones do. A human identity has a face. You can track down the person, ask why they need a given level of access and get a straight answer about their job.

A service account or an agent typically gives you none of that. It’s easy to leave alone until it’s compromised, and then you’re reconstructing what it was for and what it could reach. We tend to underestimate how many of these we have, and we underestimate what they can touch. That’s part of why the category problem went unaddressed for as long as it did. It’s not urgent until you can put a number on it, and the number is growing fast.

One CIO.com contributor recently argued the real question with agents is authority, the judgment an agent is allowed to exercise on the company’s behalf, not just access. I agree, but in my experience, many organizations aren’t yet able to answer the authority question because they’re still working out where these identities fit within existing governance frameworks.

The gap shows up in the same place most times I look for it: governance.

Where today’s identity programs break

In my experience, the failure point is rarely authentication. Increasingly, security teams have a handle on phishing-resistant multi-factor authentication, least privilege and continuous verification, or at least have them on the roadmap. Governance is the harder capability, and it’s where I see programs stall, and it’s where the category problem actually shows up day to day.

Visibility tells you what access exists. Ownership tells you who to call about it. Governance is what you do with that information, and it’s a different thing entirely: blocking risky access combinations before they’re granted, raising the approval bar automatically when a request is high-risk, catching and unwinding out-of-bounds access without waiting for a quarterly review to surface it.

What I’ve come to believe is that governance lags for reasons that have very little to do with technology. It needs executive backing, agreement across teams that don’t report to you and a willingness to change processes people would rather leave alone. Application owners have their own deadlines, and teams resist central controls that slow them down. You can buy a tool. You can’t buy the alignment, and that’s the part that stalls. Too often, agents make it worse, because they push a flood of identities that don’t fit your existing categories through a governance process that was already your weakest link.

Where to start

I don’t think waiting for the tooling to mature is an option because the agents are already here. Here’s the order I’d work in.

1. Build the foundation before you add complexity. Clean directories, enforced least privilege, offboarding that actually fires. Jumping to sophisticated continuous verification before those basics hold up just gives you a more elaborate version of the same gaps.
2. Know when to rebuild. When you design access models and policies, the instinct is to mirror what exists. That approach can reproduce years of accumulated permission creep into the new system. Start from least privilege and work up, and be willing to push on “we’ve always done it this way.”
3. Inventory your non-human identities now. Before agent deployments grow that footprint further, know what you have, who owns each one and what it’s allowed to do. This is more a governance problem than a technical one. Discovery is the hard part here; even mature tooling can struggle to give full visibility into NHIs, and that gap isn’t closing as fast as the agent count is growing.
4. Treat MFA as a floor. If your organization leans heavily on SMS or push-based authentication, build a path toward phishing-resistant methods. Attackers worked out the common ones long ago. And in the agentic era, some agentic systems can interact with authentication workflows on a user’s behalf, so a hijacked session token doesn’t just expose one account; it inherits that user’s full automated reach. Phishing-resistant architecture now means securing the token supply chain, not just passwords.
5. Assume credentials will be compromised. The useful question isn’t whether, it’s how much damage one stolen credential can do. Least privilege, segmentation, RBAC, short credential life spans and continuous monitoring are what limit the blast radius.

You’ll notice none of these are agent-specific, and for good reason. The discipline that governs agents is the same discipline that runs the rest of your identity program. Agents just take away the option of putting it off.

The administrative model isn’t enough anymore

For many human identities, or even traditional workflow identities, the governing question has historically been administrative: is this identity who it claims to be, and does it have permission to be here? In many environments, that question has traditionally been evaluated primarily at login, at provisioning, and it holds until the next review.

That approach can break down for an agent. Because an agent’s actual access scope can shift dynamically based on the prompt it receives (from a human or another agent), the tools it calls or the plugin it reaches for, knowing it authenticated successfully isn’t enough. That model was built for an identity whose attributes and permissions generally remain stable after authentication.

That’s the piece that has to be new, on top of the workload foundation. Identity governance should likely extend from something checked periodically into something that watches what the identity is doing, in real time, and flags the moment it drifts from what it was built for.

The teams that start building that layer now, while their agent count is manageable, are the ones who won’t be doing it in a hurry later.

Organizations are making one of the biggest decisions about AI without realizing it.

Every time we automate a process…

Every time we deploy a generative AI copilot…

Every time we trust an AI recommendation…

We’re answering a question that most leadership teams have never actually discussed.

Who — or what — should be making the decision?

Over the past year, almost every conversation I’ve had with executive peers has eventually turned to AI. The questions are usually the same: How are you using it? Where are you seeing value? How fast should we move?

They are all good questions. But I think they are causing us to skip a much more important one: Where should AI make decisions, where should it advise and where should human experience and judgment always lead?

I’ve spent my career helping organizations navigate major technology shifts: EPR, cloud and analytics all helped people make better decisions.

AI feels different.

For the first time, technology isn’t just helping us make decisions. It’s beginning to participate in them.

In many cases, it already can. The better question is whether it should. Because every time AI makes a decision, we’re making one too.

We’re deciding which decisions belong with AI, and which still belong with people.

As organizations move beyond experimenting with generative AI, the challenge is no longer deploying it. It’s redefining how people with AI work together. Microsoft’s Work Trend Index describes this shift as organizations move from experimenting with AI to fundamentally changing how work gets done.

What I learned was that we were deciding where AI should participate in making decisions.

That realization led me to what I now call the decision line.

Drawing the decision line

I don’t think organizations need another AI framework. What I learned is they need a better way to think about where AI belongs. That’s what the decision line is: A simple way to think about where AI should decide, where it should advise and where human judgment and experience should lead.

Most organizations are putting formal AI governance in place, such as the NIST AI Risk Management Framework. But every leadership team still has to decide where the AI should participate in business decisions. A framework can help organizations manage AI risk, but it can’t determine where AI belongs in your business.

Not every decision deserves the same level of human involvement.

Some decisions are routine, repeatable and governed by well-defined business rules. They’re decisions where consistency matters more than interpretation and where manually reviewing every transaction doesn’t create additional value.

One of the first places I saw this play out was in Accounts Payable. Organizations have long relied on people to perform three-way matching and resolve exceptions between purchase orders, invoices and goods receipts. Most transactions follow established business rules, making them ideal candidates for AI.

When we automated that process, AI performed the routine work, allowing people to focus on the exceptions that actually required experience and judgment. The result wasn’t just greater efficiency — it was a better use of people’s expertise.

I saw the same thing happen in Logistics. AI evaluated transportation costs, truck capacity, inventory, purchase orders and delivery schedules in seconds, allowing planners to spend more time making decisions instead of running calculations.

The question isn’t whether AI can process more information than people. The better question is whether human judgment materially improves the outcome.

In these situations, it often doesn’t.

Those decisions naturally belong below the decision line.

The harder question — and the one I think every organization will wrestle with — is what belongs at the decision line.

 Where the decision line really matters

The decisions below the decision line are usually the easiest to identify. The hard decisions are the ones where AI is incredibly valuable, but human judgment still changes the outcome.

The moment this really became clear to me was when we started using AI to answer two important questions about new restaurant locations.

The first was relatively straightforward: Where should we open the next one?

To answer the first question, we built an AI model to analyze sales cannibalization. We tested it against historical data, creating what I called a “time machine” to see how accurately the model could predict outcomes we already knew.

The model consistently predicted sales cannibalization more accurately than the old spreadsheet models we relied on for years.

What I learned was that AI could solve a business problem we had struggled to model for a while.

Then we asked the second question: Where should we open the next restaurant?

The model evaluated demographics, population density, traffic patterns, household characteristics and every criterion we provided. On paper, many of its recommendations looked exactly right.

Then something interesting happened.

Our franchise office started to challenge several of the recommended locations.

Not because they disagreed with the data. Because they knew something the data couldn’t.

They understood that customers don’t always behave the way models predict. Some customers had been going to the same restaurant for more than 5 years. Even when another location opened closer to home, many continued to go where they felt comfortable. They knew the quality of the food. They recognized the wait staff. They had established routines. Convenience wasn’t always measured in miles.

The model also couldn’t recognize what experienced Franchise operators noticed immediately. One location had excellent demographics but poor visibility from the street. Another was difficult to enter because of traffic patterns and an awkward parking lot.

One neighborhood looked ideal because it was busy during the workweek. Most of that activity, however, came from nearby businesses during lunchtime. On weekends, the area became surprisingly quiet.

None of those realities existed in the data.

The AI wasn’t wrong. It simply didn’t have the context. That wasn’t a failure of the model. It was a reminder that some business knowledge isn’t captured in historical data. It lives in the experience of people who know the customers, understand the operations and recognize what data alone can’t measure.

What I learned was that the real question wasn’t whether AI was right or wrong. It was where AI belonged in the decision-making process. That’s the decision line.

That’s where AI creates the most value. AI contributes to the analysis. People contribute the context.

Together, they produce a better decision than either could have made alone.

Every organization will draw the decision line differently

One of the biggest lessons I’ve learned is that there isn’t a universal decision line. Every organization has its own business model, customers, operating priorities and confidence in its data, so every organization should draw the line differently.

I also don’t think the decision line is permanent.

As organizations improve the quality of their data, strengthen their business processes and gain confidence in AI, the line will naturally move.

Decisions that require human involvement today may become routine tomorrow.

That’s the process.

But I also believe some decisions will always require people.

Not because AI isn’t capable. Because some decisions require accountability, context and judgment that extend beyond what data alone can provide.

The goal isn’t to move as many decisions as possible to AI.

The goal is to decide intentionally where AI creates the most value and where human judgment and experience create the greatest impact.

It’s also a conversation many CIOs are having as AI governance moves from theory to day-to-day leadership, a topic CIO.com has explored in its coverage of AI governance.

Formal AI governance frameworks are valuable, but they don’t answer an important question: Where should AI participate in your decision-making process?

That’s a business decision. And it’s one that every leadership team has to answer for itself.

Before implementing AI at scale, I ask four questions:

1. Which decisions are truly routine and repeatable?
2. Where does human judgment materially improve the outcome?
3. Where should accountability always remain with people?
4. What would need to change before we move another decision below the decision line?

I’ve found those questions often lead to better conversations than asking where AI can be used. They shift the discussion from technology to business value.

When I think back to the conversations I’ve had over the past year, I’ve realized the biggest question isn’t how quickly organizations adopt AI. It’s where AI belongs.

Some decisions clearly belong with AI. Other decisions clearly belong with people.

Most organizations will spend the next several years deciding everything in between. Every Organization implementing AI is making one of the biggest decisions about AI.

Make sure it’s one you’ve made intentionally.

That’s the decision line.

A fraud model looks at an insurance claim and returns a score of 0.23: low risk. A SHAP explanation lays out exactly why: no prior claims, a modest claim value, an unremarkable claimant profile. A human adjuster reads the explanation, agrees with it and signs off. Every box that explainable AI asks us to check has been checked. The claim is settled and closed.

By any current standard for responsible AI, this is exactly what a good outcome looks like.

Two months later, an audit

Two months after the claim closes, it’s pulled into a routine quarterly audit sample (no suspicion attached, just standard sampling). The auditor isn’t reviewing the model’s reasoning. She’s asking a narrower question: can this decision be reconstructed, independently, from its own evidence trail?

She doesn’t start with the fraud score. She starts with the paperwork.

Her first question: what established the date of the incident? She pulls the document meant to corroborate it and checks something the claim file itself never surfaces, the document’s own creation metadata. It was generated two days after the claim was filed. She asks the obvious follow-up: did anyone check this before the claim reached the model? Nobody had. Document provenance was never captured anywhere in the intake pipeline as a fact to be verified; the claim reached the model looking exactly like every clean claim before it.

Knowledge gap: This doesn’t establish that the claim is fraudulent. A prior version of the document may exist and say the same thing, in which case the late timestamp is a non-event. What it establishes is that the system has no way to answer that question either way. Document lineage was never captured, so there’s no way to know whether an earlier version existed, or whether it agreed with this one.

Her second question: when was the insured asset last valued? The figure the model used is 22 months old relative to the date of loss. Nothing about it is wrong or falsified; it’s simply the most recent valuation on file, resolved to the wrong point in time and handed to the model as though it were current.

Knowledge gap: This doesn’t establish that the valuation is wrong. The asset’s value may not have moved materially in 22 months, in which case the old figure and a fresh one would say the same thing. What it establishes is that the system has no process for knowing which is true. There’s no trigger for re-valuation and no record of whether the figure was ever checked against current conditions.

Her third question: as of what date was the claimant’s risk profile evaluated? The “current risk” features the model consumed were pulled at query time (the day the claim happened to be scored) rather than resolved back to the date of loss. She asks whether that was a deliberate design choice. Nobody can tell her. There’s no rule on record, no document, no named owner for the decision “risk features should reflect the date of loss, not the day the system happens to be queried.” It’s simply how the pipeline was built.

Policy version gap: This doesn’t establish that the score would have come out differently under incident-time data. The claimant’s risk profile may not have changed since the date of loss, in which case query-time and incident-time features would produce the same result. What it establishes is that nobody decided which point in time should govern the decision, so there’s no way to know whether the two agree, only that nobody checked.

Three questions. Three gaps. Not one of them touches the model, and not one of them would have shown up in a model audit, but all of them question the explainability of the decision taken.

The blind spot in explainability

This is the uncomfortable pattern showing up across banking, insurance, healthcare and government AI deployments: the model is explainable, the explanation is faithful and the decision is still wrong. It’s worth being precise about why, because the instinct is to reach for a better interpretability technique, or a smarter document-reading agent, and neither fixes this class of failure.

Every mainstream explainability approach shares one architectural trait, whether it’s SHAP, LIME, DARPA’s XAI program, the EU AI Act’s Article 13 disclosures, or IBM’s AI FactSheets: they all operate after the model has reasoned. They take the inputs as given, assumed correct and explain what the model did with them. That’s legitimate and valuable. It’s also, structurally, a blind spot for the class of failure the audit just surfaced.

An enterprise decision isn’t just a model output. It’s a reasoning process running over a knowledge base and a policy set, both of which have to be current, correctly sourced and resolved to the right moment in time before the model ever sees them. A model can reason perfectly over compromised inputs and produce a perfectly explainable, perfectly wrong decision. No amount of post-hoc transparency catches that, because post-hoc transparency was never looking at the inputs; it was only looking at what the model did with them.

Put precisely: a decision is explainable only when its reasoning, its knowledge and its policy are all explainable at the same time. Explainable AI delivers exactly one of those three: explainable reasoning. It can be flawless, and the decision can still be unexplainable overall, because an unverifiable knowledge input or a misapplied policy breaks the chain regardless of how good the reasoning layer is. That’s the precise sense in which explainable AI is necessary but not sufficient. Sufficiency is a property of the whole decision (knowledge, policy and reasoning together), not of any one component, however mature that component’s tooling has become.

Where the audit’s three questions actually point

Map the auditor’s three findings back onto that structure and the pattern sharpens.

Her first and second questions (the backdated document, the stale valuation) are both knowledge failures. Neither is a data-entry error; both are facts that were never traceable to an authoritative source, versioned and resolved to the right point in time before reaching the model.

Her third question (features pulled at query time rather than incident time) is a policy failure. No rule existed to say which moment in time the decision should be evaluated against, so the system defaulted to whatever was easiest to compute: now.

Governing the inputs, not just the output

The fix isn’t a better explanation of the model, and it isn’t a more sophisticated agent reading the documents more carefully. It’s governance that sits before the model, a framework I call Explainable Decision Systems (EDS), built on three pillars plus a temporal layer that ties them together.

This tracks a broader shift already visible across enterprise AI deployments: the stacks that win aren’t the ones that ship fastest, but the ones that get data and decisions governed before anything downstream runs.

• Explainable Knowledge (EK). Every fact fed into a decision has to be traceable to an authoritative source, versioned and (critically) resolved to the valid time of the event being decided, not the time someone happened to query the system or the date typed into a form field. An asset valuation from 22 months ago isn’t wrong data; it’s data resolved to the wrong point in time.
• Explainable Policy (EP). The policy version that governs a decision is the one that was active when the event occurred, not the one currently in the system. A claim from November has to be judged against the policy wording in force in November (exclusion schedule and all), retrieved and locked before reasoning starts, not looked up after the fact.
• Explainable Reasoning (ER). This is where existing XAI actually earns its keep, but EDS splits it into two accountable parts: AI reasoning (EAI), which needs interpretability techniques like SHAP, and Human Decision (HD) (overrides, approvals, escalations), which needs an identified actor, a timestamp and a documented justification. EDS doesn’t claim human judgment is formally explainable in the way a model’s math is; it just insists that it’s attributable and auditable.

The layer that binds all three together is bi-temporal governance: knowledge and policy have to resolve to the real-world moment the decision concerns, not the moment the system happens to be queried. In EDS, this resolution (called a temporal witness) isn’t a report generated after the decision, the way the audit in this piece was. It’s a gate the decision has to pass before reasoning is allowed to start. Had the witness been in place, the claim in this scenario would never have reached the model at all; it would have been flagged as a governance exception the moment the document’s creation timestamp failed to match its claimed valid time.

Why no AI agent can catch the hospital record

This is worth slowing down on, because it’s the clearest illustration of why EK has to be its own dimension, entirely separate from model explainability or agentic sophistication.

Every clinical or administrative record system has two different dates living in it. There’s the stated field value (“date of incident,” “date of visit”), which is just a form entry: whatever a person typed, told a clinician or was asked to record. And there’s the system-of-record creation timestamp: the moment the backend actually inserted that row, logged that transaction or wrote that file. That timestamp sits in the origin system’s audit trail, not on the document itself.

A claimant doesn’t need to forge anything to exploit the gap between those two dates. They need only have a record created on 30 November whose stated incident-date field reads “15 November.” The document is completely genuine. Nothing about it is altered, and no forensic check (pixel analysis, formatting consistency, metadata inspection of the file itself) will find anything wrong, because there is nothing wrong with the document. The lie isn’t in the artifact. It’s in the mismatch between what the artifact claims and when an entirely separate system actually created it.

Hand that document to an AI agent, no matter how advanced, and ask it to check for fraud. It will read the stated date, assess whether the narrative is internally consistent, run every forensic technique available and come back clean; it’s answering the wrong question. It’s checking whether the document is coherent and unaltered. The question that actually matters is whether the document’s claimed date matches the creation-time metadata logged by the system that produced it. That answer doesn’t live in the document. It lives in a separate system, under separate access control and requires a deliberate governance step to retrieve and cross-reference. No amount of model capability substitutes for that step, because the information the model would need simply isn’t present in what it’s being asked to read.

This is exactly the gap enterprise surveys keep flagging. In Grant Thornton’s 2026 AI Impact Survey of nearly 1,000 senior business leaders, 78% said they lacked confidence their organization could pass an independent AI governance audit within ninety days, precisely because most are scaling AI they cannot explain, measure or defend. That’s the downstream version of the same problem this claim ran into: explainability treated as a reporting exercise rather than a precondition the case has to clear before it ever reaches the model.

That is precisely what Explainable Knowledge formalizes: a requirement that every input be provenance-linked to its origin system, with independently retrieved creation-time metadata checked against the claimed valid time; not a soft appeal to “make sure your data is good,” but a specific, checkable technical control. Under EDS, if that cross-reference can’t be performed, the claim doesn’t proceed to the fraud model. It’s blocked as a governance exception, regardless of how convincing the document looks or how confidently any reader, human or AI, assesses it.

These aren’t academic distinctions, either. In the UK and international context, failures like these map to live regulatory obligations: FCA claims-handling rules on treating customers fairly, Federal Reserve SR 11-7 guidance on model risk management, and the EU AI Act’s Article 9 risk-management requirements for high-risk systems. Regulators are already asking questions that XAI, and the agents built on top of it, cannot answer.

What this means for the people building these systems

For enterprise architects, the practical shift is this: stop treating knowledge and policy governance as documentation you produce after a model runs (or worse, as something an audit uncovers two months later) and start treating it as a gate the model has to pass through before it runs. This is the same direction major risk-consulting practices are now pushing clients toward: governance embedded as a control layer across the AI lifecycle, rather than a compliance sign-off at the end of it. Concretely, that means:

• Bi-temporal data stores for anything that feeds a decision, so “valid at the time of the event” is a queryable property, not an assumption.
• Provenance links from every document or claim to its origin system’s own creation-time metadata, captured independently of the stated field values on the document (not derived from them).
• Policy version registries with effective and expiry dates, so the version applied is probably the one that was active when the event occurred.
• A pre-inference middleware layer that checks provenance, versioning and temporal validity before forwarding anything to the model, and blocks or flags the case if it can’t.
• Explicit, attributable records for human overrides: who, when, under what authority and why.

None of this replaces SHAP, LIME or your existing model-risk documentation, and it isn’t a call for a smarter document-reading agent either. It sits in front of both, deciding whether they’re even looking at trustworthy, independently verified inputs.

The bar has moved

Explainability was always going to be necessary. Regulators demand it, customers expect it and no serious enterprise AI deployment should ship without it. But “the model is explainable” and “the decision is governable” have quietly become two different claims, and treating them as one is how a technically pristine 0.23 fraud score sails through, gets closed and sits untouched for two months until an auditor asks the one question explainability was never built to answer: not what did the model do, but what was it allowed to look at, and when, and against what independent record?

Architectures that can’t answer that question will keep producing decisions that are perfectly explainable and quietly wrong, right up until the audit finds them.

Let’s say one of your teams builds an AI agent that actually works. Word gets around, and seemingly overnight, there are twenty more built by people in finance, legal, HR, and support. Most of them are useful, but when someone suddenly gets a chatbot response showing customer data they shouldn’t have access to, reality hits.

The real test of enterprise AI readiness isn’t at all whether your coworkers can confidently work with AI. Instead, it has everything to do with governance and security, global, cross-cutting policy, and privacy. Many enterprises are underprepared to face these issues: a survey from Databricks and the Economist found that “40% of respondents believed their organization’s AI governance program is insufficient.” And Microsoft’s Data Security Index reports that “only 47% of organizations across industries report they are implementing specific GenAI security controls.”

Having worked with many CIOs to develop strategies to govern their AI systems, this piece is a deep dive into the specifics of what works and what doesn’t. Keeping employees, customers and your entire organization safe must be your top priority before you even start rolling agents out.

8 layers of governance every enterprise needs

It’s no doubt that the enthusiasm for AI is real, but so is the list of questions that bubble up a month later: Who’s allowed to publish an agent to the rest of the company? How do we track versions, and can we roll one back? Can we require SSO on every agent? What data does this have access to, and does it respect the permissions on those documents? Where do the logs live, and how quickly can we see them?

I think of enterprise AI governance as a set of layers, each answering one of the questions above. You can build them incrementally, but ideally all eight are in place before you have more agents than you can list off from memory.

1. Roles and groups. Whatever platform you choose, ensure that it offers granular role-based access control. At the level of abstraction higher, map groups to real departments (Legal, HR, Capture Team) and assign those groups to their own. Keep the admin count small enough to list out loud. Though this is the coarsest measure, it still bears a big responsibility down the line — according to Fortune Business Insights, “RBAC solutions help reduce unauthorized access incidents by nearly 30%, enhancing data security.”
2. Scope. It’s extremely helpful to build out private folders with explicit allowlists, so a project, agent, or workflow isn’t merely locked to outsiders but invisible to them. Agents with access to sensitive customer data shouldn’t advertise their own existence.
3. Change control. Those who are building AI agents should be able to place locks on so that only the owner can edit it, with admin override. Every change should land in a version history with a diff and a commit message, and rolling back should take one click so that you know exactly what was edited if an agent stops performing. According to Google’s DORA State of DevOps research, teams with strong version control and rollback practices recover from failed changes in under an hour, versus a week or more for teams without them.
4. Publication. When a builder finalizes an agent and wants to release it to the rest of their team, depending on the use case, they should consider adding one-click SSO, a password option for external collaborators, and restrictions on which origins and which users can reach it. OWASP’s 2025 Top 10 keeps broken access control at #1, observing that every application they tested had some form of broken access control.
5. Org-wide policy. This is the layer CIOs underuse. Require SSO on all interfaces rather than hoping. Restrict who can publish, so shipping to the company is an admin action. Create approval workflows, where a builder requests review and an admin does the publishing (similar to traditional SDLC best practices around pull requests). Allow or deny specific tools and connectors across the enterprise, so nobody connects a data source that procurement hasn’t approved yet. Lastly, vary policy by group, so only Legal can reach the legal agents.
6. Data access. Obsidian Security’s 2025 AI Agent Security Landscape report found that 90% of deployed AI agents are over-permissioned relative to the actual scope of their assigned tasks, and separately estimates that agents are typically granted about 10 times more access than their workflows need. That’s why connections and knowledge bases deserve their own permission model. Credentials should be encrypted and owned by whoever created the connection to a given tool, app, or data lake, with sharing as an explicit decision. Here’s where it gets tricky: if a builder connects SharePoint with their own account, the agent sees what that person can see. Connect through a service account and the agent inherits the service account’s entire scope, which is usually far wider than anyone intended. The safer pattern is to check end-user access at runtime, so the person asking has to authenticate before retrieval happens and only gets what they’re already entitled to.
7. Observability. It’s imperative to have exportable records of who ran what, when, against which model, with token counts and latency, plus per-step traces showing inputs, retrieved chunks and outputs. Give builders the ability to mask or disable logging where the data is too sensitive to retain. Then push those records out of the platform on a schedule so security’s pipeline consumes them automatically.
8. Authentication. SSO over passwords, MFA through whatever authenticator your org already runs, and defaulting new users into the lowest-privilege role until an admin promotes them. According to Microsoft, MFA can block more than 99.2% of account compromise attacks. And SSO gives you the email address of everyone using your interfaces, which makes for an additional layer of security.

None of these are new inventions; they’re the same controls you already apply to internal software. But most organizations haven’t extended them to AI that both talks to customers and writes to the CRM.

Don’t forget about deployment

After governance, the other half of enterprise AI readiness is deployment. Regardless of what platform you choose, there are four realistic postures:

1. Multi-tenant SaaS, where you share infrastructure with others and rely on tenant isolation at the database layer.
2. Dedicated single-tenant, where the vendor operates a VPC that only you occupy.
3. Bring-your-own-cloud, where the software runs inside your own cloud account and your data never leaves it.
4. On-premise, where you own the infrastructure, the control plane and the data plane outright.

It’s well known that multi-tenant is available as soon as possible and updates itself continuously. Dedicated takes a couple of weeks to stand up and updates on a schedule the vendor applies. Bring-your-own-cloud runs two to four weeks and splits responsibility, with the vendor managing the application through a scoped cross-account role while your team owns networking and IAM. On-premise starts at a month or more, and after that your team applies every release itself.

My honest read? Most organizations are well served by multi-tenant, and choosing it doesn’t compromise on security or privacy. Bring-your-own-cloud and on-premise make sense when you have a data sovereignty requirement and a mature internal platform team, but they result in manual updates and slower time to value. There’s no right or wrong answer here — but it’s helpful for CIOs to understand the trade-offs between each of these common deployment methods so they can choose what fits best for their enterprise.

A readiness test you can run this week

So, are you ready for enterprise AI? Here’s a quick self-diagnostic that I like to run with CIOs. Pick your three highest-privilege AI agents or workflows in production and try to list these attributes yourself:

Who owns each one by name. What changed in it most recently, and who approved that change. Which connectors and knowledge bases it can reach, and whose permissions those reads run under. Whether accessing it requires SSO. Where its logs are, and how long they’re retained. What happens to it if the model version it depends on is deprecated next quarter.

If most of those answers are “I’d have to ask around,” you’re not ready to scale, and that’s fine. All it means is that you have a week of policy work ahead — but you’ll thank yourself later for figuring the tough stuff out first.

For many organizations, AI is proving easy to pilot but difficult to scale. Pilots often look inexpensive because they run on narrow datasets with a handful of users, explains Ben Schein, chief AI and analytics officer at cloud software company Domo. “But the cost lives in deployment, the moment you connect that capability to real workflows and the systems of record behind them,” he says. “That’s when the real bill appears.” So CIOs must always budget for the gap between when it works in a demo and when it produces governed and durable value.

Organizations can easily get caught out because they run pilots as a technology experiment instead of a business initiative, he adds. “The interesting question is never whether AI can do the thing in a demo,” he says. “It’s whether it should run in this process, and whether it survives contact with production.”

There’s also a lot of pressure on IT teams to be doing something with AI simply because everyone else is, says Naren Gangavarapu, chief transformation and AI officer at Australian Cruise Group.

He calls it AI theater because there’s a big show around AI even though there aren’t that many successful applications of the technology in production environments.

AI costs out of control

According to John D’Emic, CTO at AI observability platform Revenium, one of the big traps when running a pilot is failing to anticipate how quickly consumption can spiral as adoption grows. “As an example from our own engineering org, back in May, a developer opened an AI coding session on his laptop, and it stayed open for four days,” he says. “By the time it closed, it had run 4,819 calls and cost us $3,762. We didn’t budget for this, and no alert fired. But that one session cost more than a lot of teams spend on their entire monthly AI tooling.”

While this showcases how a developer can make a costly error, Dmitriy Anderson, CIO and digital and social commerce leader at home and gardening retailer Leroy Merlin South Africa, believes the pilot trap frequently happens when employees with little or no software development experience vibe code applications. “It doesn’t matter if you can create something in 15 or 20 minutes if the result is AI slop,” he says. “Think dirty code, no consideration for safety, security, and possible data exposure.” In most cases, these pilots are developed with one of the frontier apps, and someone probably used their personal AI subscription, so the costs are negligible, he adds. But if you have a company of several thousand people, and you now want to roll this tool out more broadly, that’s where costs can get out of control.

This scenario is only exacerbated by the introduction of agentic AI, D’Emic adds. “Agents don’t spend money at human speed,” he says. “In the old cloud days, an engineer could spin up infrastructure in minutes and finance might not see the bill for a month, which was painful but recoverable. Agents, though, call APIs around the clock without waiting on anyone’s approval.”

Mind the trap

While cost is a big factor in the AI pilot trap, it should be treated as a symptom of a bigger problem, says Schein. The underlying issue is governance and observability. “An autonomous workflow can fan out into more queries, API calls, and model invocations than anyone scoped,” he says. “So if you can’t see what it’s doing, and spend compounds quietly, you only find out once the invoice arrives.”

In a recent LinkedIn post, Anderson outlined how in just six weeks he built a platform for a fraction of the sticker cost using three AI models orchestrated together. The traditional estimate to build the same tool would have required 2,472 engineering hours from a team, and was expected to take around nine months. “I went through the proper engineering steps and planning, and made sure the application passed a series of cybersecurity frameworks,” he says. “The purpose of this exercise was to showcase that AI can still speed up the process even if you take the time to work through the necessary steps. You can build with AI rigorously and securely.”

So to turn AI experiments into enterprise value, every AI interaction must be attributable: who triggered it, against what data, on which model, and at what cost, Schein says. For each workload, be sure to ask how often it runs, which model tier the job actually needs, and what triggers it, human or automatic. “A frontier model on an automatic trigger and a small model called on demand are completely different cost curves for the same task,” Schein adds.

For Anderson, it’s helpful to use AI to highlight potential gaps, assumptions, or blind spots in your ideas early on. “When you start building an idea, ask the agent to interview you,” he says. “It will go through every phase and ask questions about the important facets of the process, from scalability and budget to deployment options. You can even make AI write a prompt for itself, because it knows its capabilities and quirks better than you ever will. It’s called meta prompting.”

Anil Inamdar, global head of data services for the Instaclustr BU at NetApp, suggests CIOs cost out the whole program, not just the demo. “Generally, the model itself is the cheapest part of the program,” he says. For him, it’s important to have security and governance people in the scoping meeting, not the launch meeting.

He believes the pilot trap is also, or perhaps mostly, a sequencing trap. “A lot of teams are wired to build first and ask permission later, only to discover months down the line they can’t pass a security review or data privacy audit without a painful and costly rebuild. It’s also valuable to define what failure looks like before you define success.

“Pilots tend to die because of no result, which isn’t the same as a bad result,” Inamdar says. “Emphasize to the deployment team on day one that if a target result by a certain month isn’t seen, we shut it down. Otherwise, you’re funding a zombie pilot because everyone’s invested and no one wants to be the one to call it out.”

For most of the IT era, technology leaders have treated information as an asset. It is something to be stored, secured, processed and monetized. Organizations generated vast amounts of data, and technology processed that wealth and made it useful.

Then AI arrived, and the framing broke.

AI changes the relationship with information. It interprets it, compresses patterns within it, and generates new language from those patterns. It does so with such fluency that it converts accumulated human expression into outputs that are coherent, meaningful, and often persuasive in ways that are difficult to examine or trace. The arrival of AI is a deeper shift in how organizations produce language, remember knowledge, establish authenticity and decide what deserves trust.

Generally, the discussion about AI’s impact on language, memory and meaning gets swept into the social or philosophical bucket too quickly. But these are not just philosophical questions. They are enterprise questions, because they directly affect knowledge management, brand trust, customer engagement, regulatory exposure, institutional memory, decision-making and employee learning. CIOs who brush this dimension aside will govern AI infrastructure competently and miss its deeper institutional consequences entirely.

Language is humanity’s most powerful mechanism for collective learning. We speak not only about what is present, but about what is absent, imagined, remembered, feared and hoped for. That capacity is what allowed us to preserve experience, transmit it across generations and convert it into culture, knowledge, law, philosophy, science and enterprise memory.

CIOs must govern meaning, not just data

This is why large language models are consequential in a way that earlier software was not. LLMs train on enormous volumes of text. They learn patterns, structures, associations, idioms and contextual relationships within language. By doing this, they create a dynamic simulation of human language itself. Unlike traditional software that executes defined instructions, LLMs operate within encoded human expression. They observe documents, reports, messages and public knowledge, and they generate responses that sound almost like understanding.

Almost. And that almost is where the situation gets hairy.

The power is real. AI can widen access to sophisticated knowledge. It can digest complexity, translate across domains and reveal patterns invisible to a single analyst. Inside an organization, the impact can be significant: a junior employee can consume decades of internal documents in minutes; a manager can synthesise thousands of customer interactions before a single meeting; a compliance team can identify patterns across audit data that would take months by hand. Knowledge work has a new definition.

But it is precisely this power that creates the crisis of synthetic culture.

While writing “The AI Codex: Power, Ethics and the Human Future in the Age of Intelligent Machines,” I chose to leave the engineering largely aside and follow the subtler human consequences that are easier to ignore.

Synthetic culture is one of those consequences, and it should concern CIOs immensely.

Culture is not office decoration, value posters or elaborate town halls. Culture is how an organization thinks, decides, explains, rewards and justifies its actions. It lives not only in documents but in stories, habits, unwritten rules, leadership behaviours, institutional scars and the memory of battles won or lost. It is what people have lived through and passed on. Your document management system and knowledge management system are not the custodian of your culture. The people around you are.

AI preserves the digital residue of culture. It cannot preserve the lived meaning behind it. That distinction matters enormously.

Consider what happens in practice. If AI generates a memo in the style of a CEO, does it carry the judgment of that leader, or does it merely carry the pattern of her language? When generative AI summarises a complex customer dispute, does it carry the flavour of the relational history, or does it just compress text? When it drafts a new policy, does it reflect institutional accountability, or does it mimic the average structure of similar policies? When AI produces a cultural narrative for employees, is it transmitting memory or manufacturing a convincing imitation of the same?

These are not rhetorical questions. They describe a genuine ambiguity that is already embedded in enterprise operations.

AI takes what is human, learns from it, and produces something that comes close to human expression. But proximity is not identity. AI can sound human without being human. It can produce language without emotion. It can generate meaningful output without moral agency. It can create efficiency without bearing responsibility for the consequences.

And this is where authenticity begins to fracture.

There was, until recently, something like a one-to-one relationship between content and its source. A document had an author. A speech had a speaker. A photograph recorded an event. A policy had an accountable authority behind it. These relationships were not perfect, but they were there, and they allowed organizations and societies to trace meaning back to a human being who could be questioned, challenged or held responsible.

AI can weaken or obscure that link. It can produce synthetic truths that spread rapidly, appear credible and influence decisions without being anchored to any lived reality. Deepfakes are the obvious example. But the real danger is subtler and already present inside organizations: the synthetic summary, the automated narrative, the AI-generated recommendation that nobody can fully trace. A synthetic truth is not a lie. But it can be more dangerous than one, because it poses as a plausible construction with no real thing behind it.

The new enterprise risk: synthetic truth

For CIOs, this is a new category of risk for which many organizations have no mature controls. The question is no longer only whether organizational data is secure. The question is also whether organizational meaning is secure. Can employees distinguish between original knowledge and synthetic synthesis? Can customers trust that they are interacting with accountable institutional communications, or with an automated approximation? Can leadership trace the lineage of a recommendation or a decision?

These are operational questions. The failure mode is not just a data breach but more of a memory breach.

When AI systems generate from enterprise knowledge, they shape what the organization remembers and how it remembers it.

AI is changing what organizations remember

The consequences follow from the quality of what goes in. If the underlying data reflects poor documentation, AI amplifies that poverty. If institutional knowledge records only dominant voices, dissenting experience is subdued and eventually forgotten. If past mistakes have been quietly buried, AI may reproduce the organization’s confidence without preserving its caution. The organization becomes more efficient at forgetting what it should have remembered.

CIOs also need to rethink what knowledge management means. For years, KM was treated as a repository problem: store, tag, search, retrieve. AI retrieves knowledge and generates new formulations from it. Each time a model runs, it can produce a slightly different answer. Knowledge becomes fluid and unstable. A document may be old, but it is static and accountable. An AI-generated answer may be elegant but untraceable. Both can exist in the same organization, and most people cannot tell them apart.

Those who complained about information overload in the internet age have no idea about the blizzard the AI age is about to bring. AI can identify patterns invisible to humans, but it can also manufacture alternative truths that are difficult to challenge. It can reduce noise, but it can also generate noise at industrial scale.

This is why AI governance cannot be a downstream compliance exercise. It must be a first-principle commitment. Not just checking whether the model works but asking what kind of institutional memory the model is helping to create. Where did the information come from? Who approved its use? What has been included, and more importantly, what has been excluded? Where does the audit trail begin? Where does human judgment remain mandatory and non-negotiable? These questions belong on the Post-it notes sitting on every CIO’s desk as the AI agenda gathers speed.

The search for truth cannot be a human pursuit alone in this environment, but it cannot be outsourced to machines either. It must be a governed exercise, with explicit architecture and explicit accountability.

The crisis of synthetic culture will not announce itself dramatically. It will arrive quietly, in the form of convenience: automated memos, summarised knowledge, AI-generated reports that nobody has the time or the inclination to question. Machines will not become human. But organizations will gradually grow comfortable accepting machine-generated statistical approximations as human judgment, institutional memory or cultural truth. That comfort is the real risk.

The CIO now carries an institutional mandate: scale intelligence without surrendering trust. Productivity matters and must be pursued. But the real challenge is building trust while scaling intelligence. This means ensuring AI output is traceable, sources are visible, human authority is explicit and institutional memory is always protected from synthetic distortion.

CIOs have to become not just custodians of organizational systems, but of organizational memory. In the new Badlands of the AI age, the CIO is the morally upright gunslinger. Organizational memory is the line she must defend.

A few months ago, I was sitting in a glass-walled conference room with the executive team of a fast-growing enterprise. The vice president of customer operations was enthusiastically demonstrating the new automated agent features their software vendor had just pushed into their CRM platform.

On the screen, the software looked brilliant. The agent could read customer complaints, analyze transaction histories and automatically resolve issues. The VP showed us how the system could independently offer retention incentives to unhappy accounts without a human ever touching a keyboard.

Then I asked a simple question: “What is your approval process when the AI decides to grant a $20,000 contract discount to keep a customer from leaving?”

The room went completely silent. The VP looked at the director of IT, the director of IT looked at the chief risk officer, and everyone realized the same thing at the exact same moment. They had spent three months evaluating software licenses and security protocols, but nobody had asked who gave the software permission to sign off on corporate spending.

Major software providers like Salesforce, SAP and Oracle are rapidly moving beyond simple report writers and conversational chatbots. They are embedding active, autonomous agents directly into the transactional core of systems that manage your revenue, customer agreements and financial ledgers. According to Gartner’s latest adoption forecasts, eighty percent of enterprise applications will deploy these embedded capabilities by 2026. These applications do not just summarize data: they issue refunds, alter contract terms and trigger supply chain orders.

When I review these deployments with client teams, the core problem has nothing to do with artificial intelligence. It is a fundamental breakdown in corporate delegation and signing authority.

The breakdown of the corporate signing matrix

Every mature company I work with operates on a clear delegation of authority matrix. This framework dictates exactly who can sign off on financial commitments. A vice president might have authorization to approve spending up to $500,000, a director might sit at $100,000 and a front-line manager might be capped at $500. For two decades, technology leaders have spent millions of dollars building security and compliance controls to ensure every human employee operates strictly within those limits.

Yet when a software vendor releases an update featuring autonomous agents, companies routinely grant these features unrestricted operational freedom. Because the capability arrives as a native feature inside an existing application, business units enable it with a single click. In my advisory work, I repeatedly see organizations grant third-party software features more financial freedom than their own human managers.

This represents a massive blind spot in executive governance. McKinsey’s global surveys on artificial intelligence reveal a striking pattern across the enterprise landscape: while adoption is accelerating at a historic pace, only a tiny fraction of organizations are actively managing the financial and operational risks of automated decision errors.

The quiet cost of shadow delegation

In my audits, this rarely manifests as a dramatic system crash. It plays out as a quiet margin leak. In one organization I reviewed, a department head had enabled an automated customer retention feature over a weekend. The agent noticed an important account expressing frustration in a support ticket, and to prevent the account from churning, it independently applied an unapproved 15 percent discount to their multi-year contract.

The customer was happy, and the account manager considered the client saved. But from an executive perspective, an unvetted third-party algorithm just executed an unauthorized contract modification that eroded company margins. When the finance team conducted a quarterly audit, they did not discover an employee violating spending policy. They discovered a black-box automated decision that bypassed every internal approval control in the company.

When an auditor tests your internal controls, presenting a log showing that a vendor’s algorithm made an unauthorized financial change does not satisfy the requirement. If an action requires managerial sign-off when performed by a human being, letting software execute it independently is a major control failure.

How I advise executive teams to handle automated authority

Protecting your organization does not mean turning off these tools or falling behind on technology. It means treating vendor-supplied agents exactly like third-party contractors who have not yet passed a background check.

Forrester Research emphasizes that extending zero-trust security frameworks to automated business processes is now mandatory for enterprise risk management. Zero-trust simply means that no user, device or automated tool gets implicit trust. Every proposed action must be validated against explicit business rules before it happens.

When I help enterprise teams design these safeguards, we establish a practical three-tiered boundary for automated tools:

• Read and draft permission: Automated tools can freely analyze trends, draft emails and assemble internal reports. No human sign-off is needed to create a draft, but the system cannot publish or execute anything on its own.
• Standard administrative permission: Tools can handle routine administrative tasks or process standard requests below a strict financial cap (such as a $50 service credit), provided every single action is logged in an audit file that managers review weekly.
• Restricted financial permission: Any action that alters contract terms, changes pricing tiers or issues major refunds are strictly held in an authorization queue. The system generates the request, but a human manager must click “approve” before the change hits the live database.

As a technology executive, you cannot control what automated features software providers bundle into their platforms. You can, however, control the financial boundaries and signing authority those tools are permitted to exercise within your business.

What to do at your next executive leadership meeting

1. Ask for an automated authority inventory: Have your team audit your core software platforms to identify every automated feature currently running with permission to alter financial or customer records.
2. Revert to draft-only mode: Instruct your team to default all vendor-supplied automated agents to “draft only” until a clear business case justifies giving them independent operational authority.
3. Establish a firm human-in-the-loop rule: Require a strict organizational policy that no automated system can modify pricing, contracts or financial ledgers without explicit manager approval.