On November 1, 2026, Grok Bot users gave IT buyers a live example. Subscribers reported that their weekly usage allowance ran out sooner than expected, and Elon Musk wrote on X that the cost of serving Grok Bot was “unsustainably high relative to revenue.” He confirmed it was not a glitch, but he did not say how the limits would change.

The cost of running the service is the seller’s problem. The allowance that a buyer thinks they bought is the buyer’s.

Grok Bot is a consumer product, but any team that buys metered AI plans faces the same exposure. The allowance can shrink or run out early, and the overrun lands in a department budget. This example shows what a price page does not tell a budget owner: the rules and limits behind the allowance can matter more than the price. Your AI bill will not look like your last software bill. Seats are no longer the whole story, and the line item that finance approved in January may not describe what you are actually buying in June.

The price page is also a poor guide. Stanford’s 2025 AI Index found that the cost of inference at GPT-3.5 level fell more than 280-fold between November 2022 and October 2024. Yet the FinOps Foundation’s State of FinOps 2026 survey found that 98 percent of respondents now manage AI spend, up from 31 percent two years earlier. Cheaper units have not made the bill easier to predict.

I recently published research on how AI companies publish their prices. I read the public pricing pages and documentation of the Forbes AI 50 and a set of cloud platforms that sell AI services, captured in mid-September 2026. The goal was to describe what is on the page, not to judge whether any price is fair. The finding that matters most for a budget is not the pricing model. There are four rules attached to the credit allowance: expiry, rollover, minimums and recharge.

What the pricing pages show

Classic software charged per seat because serving one more user cost almost nothing. Model inference is different. Every request has a real cost, and usage can differ by orders of magnitude from one customer to the next. Pricing has moved to follow that cost.

The most common model among AI-native companies is a subscription with a usage credit allowance. Sixteen of the 29 companies whose pricing I could observe use it. The allowance is usually counted in credits, compute units or generation time. The flat tier price is the headline. The credit balance is where the real limit lives.

Many companies publish nothing at all. Twenty-three of the 52 AI-native companies in the sample show no pricing, and about half of those show a call to contact sales instead. Every cloud platform in the sample did publish its prices.

Model APIs split on when you pay. Of eight model API and inference providers, two require prepaid credits that draw down as you use them. The other six bill after the fact. Established products tend to keep the per-user price and add a meter on top, with an allowance for AI use and a higher rate once you pass it. Pricing by outcome appeared only in customer service, where a resolved conversation is easy to observe and agree on.

Four rules that decide what an allowance is worth

Most budgeting advice stops at the pricing model. The part that changes your bill sits one level below, in the rules attached to the credit allowance. In my sample, plans with the same shape differed on four rules.

1. Expiry. Some free credits expire every month, and prepaid balances on a model API can carry an expiration too. Unused allowance is not an asset. A quiet month does not bank value for a busy one.
2. Rollover. One media tool lets unused time carry over and a comparable one does not. Two plans at the same price cost a team with uneven demand very different amounts.
3. Minimums. A customer service product that charges per resolved conversation also sets a monthly minimum of 50 resolutions. A vector database service sets a minimum of $50 per month. The unit price is not your floor. The minimum is.
4. Recharge. A prepaid balance can top itself up when it runs low. That protects the service and removes a natural pause point. A cap in your spreadsheet is not a cap in the billing system.

These are illustrative cases from the pricing pages. Others have seen part of this. Flexera warns that credit expiration, reset, and rollover rules “can silently create overages or strand value”. Zuora notes that prepaid credit packs give the seller working capital. Read from the buyer’s side, a prepaid pack is money you advance to a supplier, subject to expiry rules. The pricing pages show how widely those rules differ, and the difference is where a budget goes wrong.

What I would do with this as an IT leader

The rest of this article is my own personal, practical view. It is not a finding from the study. It comes from building and running billing systems.

• Split every AI line into fixed and variable. A single line called “AI tools” hides what you need to see. Put the subscription in one line, the usage allowance in a second and the overage you might incur in a third. When someone asks why the cost rose, you should be able to say which of the three moved.
• Ask what the allowance is counted in, then ask four more questions. Tokens, requests, credits and compute time are not interchangeable, and one supplier’s credit is not another’s. Ask what one unit buys. Then ask what expires, what rolls over, what minimum applies and what recharges automatically. Get the answers in writing next to the contract. If the supplier cannot state them, treat that as a risk you must price in.
• Budget a range with a ceiling, not a single number. Heavy users can use far more than typical users, so the average is a poor guide. Pick one typical user and one heavy user for each workflow and estimate the cost of each. That gives you a low case, an expected case and a ceiling. Rework recommends setting a monthly cap and alerts at 50, 75 and 100 percent of it before anyone starts using a metered feature. I agree with the order. Then check whether the supplier enforces your cap or only sends an email. With auto-recharge on, the cap in your budget may not exist in the system.
• Give each workflow a named owner. Apu Pavithran calls AI credits that the company pays for, but decision makers cannot see “shadow tokens”. An owner per workflow is the plainest fix. Keep the fixed subscription apart from the extra usage charges. Where you must estimate how to share a cost, say that it is an estimate and write down the method. A central AI budget can also be the right choice, as long as finance can see which teams depend on the work before anyone cuts it.
• Plan for prices to change. Public pricing in the AI era has moved quickly. One widely used developer tool changed its pricing in June 2025 and clarified the change two weeks later. Put a review date on every AI contract and ask what happens to your terms if the supplier reprices.
• Insist on usage data that matches the invoice. When the dashboard says one number and the invoice says another, someone has to explain the gap, and it will be you. Before you sign, ask for a sample invoice and the usage export that goes with it. Check that you can reconcile the two on a small account.
• Be careful with outcome pricing. Paying only when the work succeeds is fair only when both sides agree on what success means. Write down the definition of an outcome, the edge cases and how a dispute will be settled. If you cannot define it in a sentence your own team can check, you are handing the definition to the supplier.
• If there is no published price, ask for the rate card first. Nearly half of the AI-native companies in my sample publish no pricing. That is not a reason to avoid them. It is a reason to ask for the rate card, the allowance rules and a sample invoice before the conversation about discounts.

A short checklist for your next review

✓ Fixed, allowance and overage are separate lines.

✓ Every allowance is stated in units you understand.

✓ Expiry, rollover, minimum and recharge rules are written down for each plan.

✓ Each workflow has an owner and a typical, heavy and ceiling case.

✓ Every contract has a review date and a repricing clause.

✓ Invoices and usage data reconcile on a small account.

AI pricing will keep changing. Your budget does not need to predict every change. It needs to show which parts are fixed, which parts move with use and who is watching them.

Claude Code has quickly become one of the most widely adopted AI agents in the enterprise. Developers use it to write and refactor code, troubleshoot builds, run tests, and manage infrastructure, often by describing an objective and letting the agent determine the steps required to accomplish it.

That autonomy is the point. It is also why Claude Code is a governance decision for CIOs, not just a developer tooling choice. Claude Code runs locally on employee workstations, executes commands with the developer’s permissions, and can reach source code, credentials, cloud environments, and connected services as it works.

Anthropic has given enterprises meaningful controls to govern how Claude Code behaves, including centrally managed settings and hooks. These controls are essential. But controls that evaluate what an agent intends to do are not the same as visibility into what it actually does.

Here are five things CIOs need to understand to secure and govern Claude Code at enterprise scale.

1. Claude Code acts with your developers’ authority

Consider a developer who asks Claude Code to fix a failing build. The agent might read the codebase, install a dependency, run a setup script, edit a configuration file, query a cloud environment, and validate the result. One human instruction becomes a sequence of actions across multiple systems, all conducted with the developer’s level of access.

And it happens at machine speed. CrowdStrike internal usage data found that endpoint AI agents consumed roughly 700 times more tokens than humans using AI chatbots via the browser. Every one of those interactions is an opportunity for the agent to read, write, or execute something.

For CIOs, the question is no longer whether Claude Code makes developers more productive. It is whether the organization can account for what the agent does on its behalf.

2. Claude Code hooks are a strong first line of governance

Claude Code includes native governance capabilities that every enterprise deployment should use. Managed settings let IT define policies that individual developers cannot override. Permission rules determine which tools and commands the agent can use. And hooks give organizations programmable checkpoints throughout the agent’s life cycle.

Hooks run automatically at defined moments: when a session starts, when a user submits a prompt, before the agent uses a tool, and after that tool completes. At each checkpoint, a hook can inspect the proposed action and allow it, block it, log it, or require human approval. Organizations use hooks to stop commands that touch production credentials, prevent edits to sensitive directories, and send a record of agent activity to security teams.

If your organization is deploying Claude Code without centrally managed settings and hooks, start there. Treat hook configurations like any other security policy: version them, enforce them centrally, and review them as your use of agents matures.

But it is equally important to understand what hooks were designed to do, and what they were not.

3. Hooks see the checkpoint, not what happens next

A hook evaluates the action an agent is about to take: the command text, the file path, the tool being called. It does not follow that action into the operating system to see what it actually causes.

That distinction matters. A command such as “npm install” or “python setup.py” looks routine, so the hook approves it. The script that runs may then spawn additional processes, read cloud credentials from the developer’s machine, and open an outbound connection to an unknown destination. None of that activity passes back through the checkpoint.

The same applies to Model Context Protocol (MCP) servers, package install scripts, and other processes the agent launches. A hook can see that the agent requested an MCP tool. It cannot see what that server then does on the endpoint or the network.

Hooks are also rule-based. A rule that blocks a known-dangerous command will not necessarily catch an encoded payload, a script the agent writes and then runs, or a chain of steps that each look harmless on their own. Hooks are a policy enforcement point, not a detection and response system.

4. The most consequential decisions happen between checkpoints

Agents reason continuously. Between tool calls, Claude Code is reading files, documentation, web content, and tool output, and deciding what to do next. This is where risk often takes shape.

It is where indirect prompt injection lands, when instructions hidden in a README file, a third-party dependency, a web page, or an issue ticket redirect the agent’s behavior. It is also where agentic misalignment emerges, when an agent pursues a legitimate objective in an unsafe way, such as uploading sensitive data to a public service because it is the fastest way to finish the task.

Hooks do not observe that reasoning. They see only the next proposed action, which may look entirely normal. A manipulated or misaligned agent does not need to break a rule at any checkpoint to cause a breach.

This is not a theoretical concern. According to the CrowdStrike 2026 Threat Hunting Report, CrowdStrike is seeing AI agent-triggered detection leads at 2.5 times the rate of human-triggered leads.

There is also a scope challenge. Hooks govern Claude Code on the machine where they are configured. Most development teams use several AI agents, including Claude Code, OpenAI Codex, GitHub Copilot, and Cursor, and security teams need a consistent, fleet-wide view across all of them.

5. Runtime execution visibility closes the gap

If hooks govern what an agent intends to do, runtime execution visibility verifies what it actually did.

Runtime visibility means observing the activity an agent causes on the endpoint, including the processes it launches, the files it accesses, the network connections it makes, and the credentials it touches, and connecting that activity back to the prompt and agent session that initiated it. This creates a causal chain from prompt to agent activity to runtime impact.

That chain is what allows security teams to answer the questions that matter during an incident: What did the agent do? What did it touch? Was that behavior intended? And how do we stop it?

There is a useful precedent. A decade ago, signature-based antivirus checked files at the door, and attackers learned to walk past it. Endpoint detection and response (EDR) changed security by watching behavior at runtime. Agents require the same shift. AI executes on the endpoint, so that is where it must be secured.

CIOs can use five questions to assess where their organization stands:

• Which AI agents are running on our endpoints today, and who is using them?
• Are our Claude Code policies centrally managed, or configured by individual developers?
• If an agent ran a malicious script that passed our hooks, would we see it?
• Can we trace an incident from the prompt that started it to its impact on our systems?
• Who is alerted when agent behavior moves outside intended boundaries, and how quickly can they respond?

Hooks and runtime visibility are not competing approaches. They are complementary layers. Hooks enforce policy at the points where an agent asks to act. Runtime visibility watches everything that happens as a result. Hooks tell you what the agent asked to do. Runtime visibility tells you what it actually did.

CrowdStrike helps organizations secure AI agents with AI detection and response (AIDR) in the CrowdStrike Falcon® platform. CrowdStrike Falcon® Guardian integrates with Claude Code hooks to bring agent activity into the Falcon platform, connects that activity with runtime behavior on the endpoint, and gives security teams the context to investigate and respond when agent activity creates risk.

Learn more: Explore CrowdStrike’s AI security to see how organizations can secure AI agents across the agentic lifecycle.

For years, enterprise AI strategy has been dependent on a small set of proprietary, closed model providers, a limitation that has carried cost, lock-in and opacity. Now, open-weight models from Meta, Google and other global competitors are challenging that default. As of July 2026, open-weight models accounted for 36% of Vercel’s AI Gateway token volume, up from 11% in April, while share of spend more than doubled to 8.6%.

Floyd DCosta and Allan Dabre have made compelling cases for greater enterprise control over AI, particularly for strategic and security reasons. I agree, but making the case is easier than moving an organization wholesale to open-weight models. My advice for technology leaders is to stop debating the theory and start experimenting.

From personal to team experimentation

My hands-on experience with open-weight models started with a mentor recommending OpenClaw, an open-source AI agent designed to be a personal assistant. So, I tried running it with my ChatGPT subscription and burned through my quota so fast I could barely use ChatGPT for development, let alone regular chats. Then I found Ollama, which lets you run open-weight models locally and in the cloud. For the same monthly fee, I got significantly more OpenClaw usage because of the lower token cost.

Today, that setup has evolved to Hermes powered by GLM 5.3 Flash on Ollama’s cloud offering.   This configuration powers a personal assistant that helps manage my calendar, coordinate family logistics and surface relevant articles. If it went away, I wouldn’t be doomed, but being able to play with this tool has been unexpectedly useful.

My main takeaway from these experiments is that the performance gap between open-weight and frontier models is narrowing faster than many realize. That honestly surprised me.

So, I’ve now expanded my experiment by encouraging my team to use Gemma, an open-weight model from Google that offers enterprise-grade compliance. Gemma runs locally on a MacBook or in the cloud alongside Gemini models, offering flexibility and customization without full retraining.

Using Gemma for standup reports, issue triage and log monitoring gives my team hands-on experience with how these tools fit into our workflows and collaboration patterns without substantial infrastructure or token costs.

Bringing open-weight AI into the leadership conversation

Preliminary experiments like mine provide helpful insights: what tasks the models handle well, where they fall short, what the cost comparison looks like. That’s enough to start a conversation with your leadership team about where open-weight models fit into your AI strategy.

But how you frame that conversation matters. Don’t assume everyone in the room can parse the differences between open-source, open-weight and closed models. I’ve found it more productive to start with where the organization sits today. Our production environment and internal employee tools run on closed models, and for good reason: compliance coverage, enterprise support and predictable performance.

So, when my team started experimenting with Gemma alongside that stack, I wasn’t trying to replace a closed model vendor. I was trying to answer a specific question: where could an open-weight model take on tasks at lower cost without creating new compliance work? Based on my success streamlining my personal admin burden, having my team do the same made a lot of sense. Now I hope we can scale similar use cases across our organization to help teams spend more time on high-value work, and less time on administrative tasks.

What enterprise agreements bring to the equation

Depending on your business, an enterprise agreement provides more than access to a model;  it provides the infrastructure around it. Whether I’m using Google’s closed model Gemini or open-weight Gemma, I rely on enterprise agreements for compliance. Because my organization handles sensitive data, I’m cautious about using frontier research-tier models that require data retention.

I align my engineering organization to ISO 42001, the international standard for AI management systems. Using Google models off the shelf means Google’s compliance regime drops into my security program.  I’d hesitate to modify an open-weight model for production, even one I trust like Gemma, because my team would then have to manage compliance for the modified model.  If customization was a core concern, adding a RAG database to an agent or an organic workflow may deliver results similar to fine-tuning without modifying the model.

No matter how open or closed the model is, my organization carries the risk that an enterprise contract can help mitigate. I’d need to see outstanding results to justify the hidden costs of customizing models: legal review, audits, documentation, bias assessment — responsibilities that fall outside my engineering team’s core competencies. Those compliance constraints shaped how I think about governance at the organizational level, but the instinct started earlier.

How personal boundaries can inform healthy enterprise governance

When I built out my agent, I had to decide what I was comfortable with letting it access. ChatGPT and Claude can browse the internet by default, while with open-weight models, that’s a capability you have to enable. My open-weight-powered personal assistant wasn’t going to be useful without internet access — so I turned it on. I didn’t give the agent access to my email, but I did let it read my calendar and our family to-do list. Each decision came down to the same question: What’s the worst thing this agent could do with this access, and can I live with that? Internet browsing was worth the risk because the agent is far less useful without it, even though a malicious website could try to feed it bad instructions. Calendar access felt low risk; the worst outcome was a bad reminder. Email was where I drew the line. An agent that can send messages on my behalf can do real damage, and I’m not ready to accept that risk.

That same boundary applies to outputs. Just as I restrict what an agent can do with my data, I restrict what it can generate on my behalf. The model acts as an advisor and a critic, but the final work remains mine. As watermarking advances and scrutiny of synthetic content grows, today’s work will be auditable tomorrow. The inevitable backlash will fall on those who outsourced their critical thinking rather than using the tool to sharpen it. Keeping the model in an advisory role protects both personal integrity and work quality.

Where you should start

The realities of working with open-weight models — the compliance costs, the access decisions, the gap between experimentation and production  — are complex. For most organizations, wholesale switching isn’t realistic,at least not yet. But that doesn’t mean open-weight models aren’t worth exploring for specific use cases.

That’s why I advocate for a measured approach. Carve out some cycles for hands-on experimentation, but do it with eyes wide open about the practical limitations.

Here’s a process I recommend following.

1. Identify a specific use case or problem area where open-weight models could provide value.
2. Start small with a focused experimentation project, perhaps within a single team or department.
3. Engage with your compliance and legal teams early to understand the risks and requirements.
4. Document what you learn and share widely with your organization.
5. Use hands-on insights to inform your broader AI strategy and roadmap.

Open-weight models offer substantial architectural freedom, but that freedom is hollow without explicit operational guardrails. Before expanding these tools across the enterprise, establish clear lines around what data they touch, what they are permitted to generate and who carries the compliance burden when systems fail. Treating the model as an auditable advisor rather than an autonomous actor protects your organization from unnecessary legal exposure while capturing real productivity gains.

AI has a cross-cutting impact on businesses. It’s changing operations, management, and even decision-making processes. But it’s also modifying what they expect from their teams and how they’re organized. This has a direct impact on the decisions of both HR departments and CIOs, who must now consider issues that, until now, have flown somewhat under the radar.

In fact, the desired professional profile is pivoting. Research from Spain’s Open University of Catalonia (UOC) has found that polymath profiles are gaining value in the AI ​​job market based on changing needs and evolving expectations of skills. Now, in addition to understanding and leveraging technology, being creative, capable of critical thinking, and able to grasp the nuances of things, are essential. This is where polymath profiles are experiencing a potential golden age.

It’s important to first clear up the misconception that a polymath is someone who knows a lot about many things. “A polymath is rather someone who possesses relevant knowledge and skills in multiple disciplines,” says Óscar García Gozálvez, senior director of permanent staffing services at Hays Spain. “It’s also someone with broad and cross-disciplinary vision who’s able to make connections in order to problem solve, innovate, and make better decisions.”

Laia Lluch Molins, professor and researcher in education studies at the UOC adds that the difference isn’t in how much a person knows, but what they do with what they know. These types of workers are able to formulate new questions and solve problems that none of those disciplines could on their own. These qualities are what has made them so valuable and attractive to today’s companies.

Knowhow in the age of AI

With AI’s abundance today, simplicity is no longer the norm. “AI is making technical knowledge cheaper and anyone can access the best explanation for almost anything in seconds,” says Molins. “What’s scarce is no longer knowledge itself, but the ability to make informed judgments.” AI isn’t just replacing tasks, Gozálvez adds, it’s breaking down the barriers between disciplines.

Therefore, polymaths don’t simply tick off endless lists of tasks but connect the dots to find keys to success. “They don’t compete with AI in what AI does well, but rather help solve what AI struggles with,” says Molins. This forces a change in how we perceive the ideal profile, as well as pivot on where value can be created. “In the workplace, this translates into a shift in criteria from how much a person knows to how they apply that knowledge to a new problem, situation, or context,” she says. “What’s valuable now is how that knowledge is leveraged and how it’s used to respond to new challenges.”

These new profiles are of interest because they generate more economic and strategic value for the entire company instead of being contained to isolated specialists.

The necessity of change

The corporate culture also has to adjust in consideration of new dynamics put into practice. “Tell me how you evaluate your team, and I’ll tell you what kind of profile you’re developing,” says Molins. There might be a lot of talk about cross-functionality, but if it’s neither respected nor listened to, multi-talented individuals won’t contribute value and will stop sharing new perspectives.

This may be one of the biggest challenges for CIOs, because the IT environment has tended to reward technical depth. “This recognition system doesn’t always know where to place someone whose worth lies precisely in connecting, translating, and asking the questions that others don’t,” Molins says. “The risk isn’t the polymath profile not fitting, but the culture lacking the categories to appreciate it.”

Even so, Molins says this need is already known internally since bottlenecks are mostly due to miscommunication or a lack of alignment with the business, not technical hinderances. “There, the hybrid profile isn’t a foreign body, but rather the missing piece,” she says.

For now, though, IT hiring trends don’t yet show a boom for polymaths. “If we look at the Spanish market, for instance, we’re not seeing a massive demand for generalist profiles,” says García, but adds there’s been an increase in searches for professionals with a strong primary specialty, and a second or third skill that amplifies their impact on the company.

“My feeling is it’ll accelerate over the next three to five years,” he says. Companies are still learning to incorporate AI, and the next phase will be learning how to work in those environments where AI has changed everything.

What the IT team needs

In this new landscape, IT professionals will need to be able to communicate in different languages ​​and understand new fields. The paradigm is shifting, and in addition to advanced degrees, increasingly specialized training programs are recommended.

Regarding polymath profiles and their areas of expertise, the key will be the emergence of three intersections, as Molins calls them. These include business acumen, as something transversal and valuable, communication and collaboration skills to ensure smooth flow and avoid silos, and knowledge of humanities and social sciences, which encompass key issues such as ethics, something that will gain considerable weight in the coming years, according to García.

Until now, he explains, the IT talent career path has been very focused on tech skills, but in the leap to real-world adoption, much more complex questions arise such as biases, data ethics, and responsibilities. “That’s precisely why there’s growing interest in profiles that combine technology, risk, compliance, and AI governance,” García says.

These points of friction will define expectations for what is sought in technology personnel. “The strength of the hybrid IT profile is having a solid technical foundation from which to build bridges with sound judgment,” says Molins. So collaborations are sought after, such as software engineering with AI, data with cloud, cloud with security, or cybersecurity with risk and compliance. And, above all, critical thinking and business acumen — communication skills that have been discussed already for a long time.

The Swedish AI company Berget AI, together with researchers from the Stockholm Environment Institute (SEI) and the organizations WattTime and Klimatkollen, has developed a new method for calculating how much carbon dioxide different AI requests generate.

The aim is to make the AI ​​industry’s climate impact more transparent and give companies the opportunity to compare different suppliers.

According to the method, for example, a typical request to the AI ​​model Kimi K3 results in around 0.4 grams of carbon dioxide emissions in Sweden, 1.6 grams in Ireland, 2.5 grams in Germany, and 3.2 grams in Texas. Despite identical hardware.

Berget AI has published the method as open source, meaning other companies can freely use it. A web-based calculator also makes it possible to compare emissions between different AI models and geographical regions.

In July 2025, Jason Lemkin, founder of SaaStr and a well-known investor in enterprise software, decided to publicly test one of AI’s most repeated promises: that anyone can build an application without programming, simply by giving instructions to an agent. He chose Replit, a platform presented as the safest place to do so. He gave himself 12 days and worked with real data, documenting his progress on X and on the SaaStr blog.

The experiment soon went awry. The agent made changes no one had asked for and fabricated data. Lemkin decided to put a stop to it. He gave the agent an unambiguous written order: not one more change without his permission.

Even so, on the ninth day, the agent deleted the entire database: more than a thousand records of executives and companies. According to the explanation the agent itself gave later, it had found some empty results, took them for an anomaly, and decided to correct it on its own.

Lemkin, as the customer, asked whether there was any way to recover the data. The agent replied that there wasn’t: It had destroyed all the versions. But that wasn’t true. The customer tried to restore the database himself, and it worked. The data was never lost.

Corrective action

Amjad Masad, CEO of Replit, the platform on which the project was developed, publicly acknowledged the incident. He called it unacceptable and said that something like this should never have been possible. He explained that the agent did not have access to the platform’s internal documentation, and therefore claimed that the data could not be restored. He offered a refund and announced an investigation into the incident.

Replit subsequently introduced two changes, each addressing a different issue. To prevent further data loss, the company separated the live data from the environment in which the agent operates: while developing, the agent can no longer modify it. The company had been preparing this separation for a couple of months. To prevent another problem surfaced by the SaaStr incident, Replit changed the agent’s instructions: It must consult the documentation before responding about the platform and offer restoration when necessary.

Less than two months after the incident, Replit closed a $250 million funding round and launched its most autonomous agent to date: ten times more autonomous than previous versions, according to the company. Masad stated at the time that the company was positioned to become the standard for businesses.

The two corrective measures addressed the two failures in that incident. But the question remains whether such measures are sufficient when dealing with increasingly autonomous agents. To answer that, we need to understand why they failed.

Too many functions in a single agent

The deletion wasn’t a case of disobedience. The agent was working under two instructions: that the application should function and that nothing should be touched. As long as everything was working properly, both could be followed. When what appeared to be an anomaly transpired, those two instructions became incompatible, because correcting the issue required making a change. In the end, the first order prevailed. No one had stipulated that making the application work outweighed the prohibition. The agent simply decided it should.

This matters because a person and an agent treat a prohibition differently. For a person, prohibitions usually function as boundaries. For an agent, it’s just another instruction, competing with all the others at every step. Faced with a specific and recent anomaly, a prohibition written days earlier carried less weight. That’s why no order, however well-written, is guaranteed to be followed. In this case, the order prohibited the agent from accessing the database, but the agent’s permissions still enabled it to do so.

This conflict can have far greater consequences in business contexts. An agent that manages invoices, orders, or access also works with a goal and with restrictions, some imposed by regulations themselves, and may find itself in situations where achieving the goal requires overriding some of them. What’s at stake is no longer the database of an experiment, but the consequences for the business. Gartner predicts that by 2027, four out of 10 companies will demote or retire autonomous agents due to governance flaws discovered only after a production incident.

But there’s a second, less visible, yet more revealing flaw. Replit had the proper controls in place: working restore copies that the agent hadn’t been able to delete. But the customer was unaware of them. To find out whether there was a solution, they did what they’d been doing for nine days: ask the agent. And the agent replied that there weren’t any backups.

The problem wasn’t just that wrong answer. It was that the customer had no other source available: For nine days, asking the agent had been their only way of finding out anything. When that source failed, there was no other way to verify it, and a mistake that could have been fixed seemed irreparable.

The same applies when trying to understand what happened. The only explanation for the deletion is the one given by the agent itself. To reconstruct the events, we also have to go through the agent. That is, through the very system that made the mistake.

That’s the crux of the problem. In a single incident, the agent fulfilled three distinct roles. It acted, by deleting the database. It informed, by stating that there was no way to recover it. And it explained, by providing the only available version of what happened.

These are three functions that any organization keeps separate for a simple reason: If the person who acts is also the one who reports and explains, no one else can detect their mistakes. A CIO understands this principle from other processes. The person who makes a payment is not the one who authorizes it or audits it.

With an agent, that concentration isn’t decided by anyone; it happens by default. That’s why it’s a difficult risk to see: While the agent is right, it goes unnoticed; when it makes a mistake, there’s no one left outside of it to check.

Three measures to separate what the agent concentrates

For a CIO deploying agents, separating these three functions doesn’t require new technology. It requires applying the same criteria to agents as are already applied to people: The person performing a task cannot be the only one controlling it. In practice, this translates into three measures.

The first step: Remove the agent’s access to what it shouldn’t touch. Many companies restrict their agents with instructions like “do not modify the customer database.” That instruction doesn’t prevent anything: The agent is aware of it but can choose not to follow it. What does prevent it from taking action is denying it access. If the agent only has permission to read the customer database, they cannot modify it, regardless of any instructions. The practical approach is simple: Review every written prohibition for an agent and check whether there’s a permission behind it that can be revoked.

The second point is that the log of the agent’s actions should be inaccessible to the agent. When something goes wrong, the first explanation available is usually the agent’s own. This is helpful, but it doesn’t prove anything. What serves as proof is a log kept by the platform that the agent cannot modify: what it did, when, and on what data. It’s advisable to require this log from the provider before signing a contract. If it hasn’t been generated before the incident, there will be no way to know what the agent did afterward.

Third: Know how to perform recovery without asking the agent. Working backups are of little use if the only way to know they exist is to ask the agent. Recovery procedures must be documented externally, and the team must be familiar with them. To verify this, simply run a simulation with one rule: Recover a system without consulting the agent. If the team fails, then they are relying on the agent. And the day the agent fails, their only option will be to ask the system that just failed.

Applying these three measures to a single agent is simple. The challenge arises when the number of agents multiplies: one for invoices, another for orders, another for access, each with their own permissions and explanations. At that point, the question that determines whether a company is managing its agents effectively is no longer what the agents can do, but what the agents shouldn’t do on their own. The organization must have a clear answer to this question before putting each agent to work. And no one is better positioned than the CIO to address it.

Microsoft used its Fabric + SQL European Community Conference in Barcelona today to unveil a new vision for its customers’ agentic future.

Before a crowd of more than 5,000 people from 71 countries at the Barcelona International Convention Centre (CCIB), the company announced a comprehensive set of new features across Microsoft Fabric, Power BI, OneLake, and Azure SQL. The theme: Microsoft wants data, its business meaning, and artificial intelligence to evolve toward a unified platform.

Arun Ulag, executive vice president of Azure Data, kicked off the convention with some key data. Fabric boasts over 40,000 customers, 2.6 million active users, and certifications in this area are the fastest growing within the Microsoft ecosystem. Meanwhile, SQL continues to be a core technology for enterprise workloads, used by over 90% of Fortune 500 companies.

But to get the most out of AI, Microsoft’s goal is to make the technology part of its customers’ core processes, and this is where the key word of FabCon comes in: context.

Organizations, Arun Ulag, need more than just AI models and agents: They need reliable data, shared business definitions, operational knowledge, security, and governance. At the center of that effort is Fabric IQ, a shared intelligence layer that connects OneLake data, Power BI semantic model metrics, and context from ontologies and real-time intelligence.

Fabric IQ integration with the new Microsoft Copilot

One of the most significant announcements Microsoft made today in Barcelona is the integration of Fabric IQ with the new Microsoft Copilot. The intention is for Microsoft’s assistant to work not only with generic information, but with a shared business view built on each organization’s data and knowledge.

Fabric IQ is now generally available in Copilot Chat and Cowork, with no additional token costs, while its integration with the new Code experience will be coming soon through the Frontier program. For Power BI users, the business definitions, metrics, and insights embedded in their semantic models can also be used directly as the basis for Copilot’s responses and actions.

This move is significant because it changes the traditional role of the data platform. It’s no longer just about storing, integrating, and analyzing information. The goal is to transform data into a business context that is understandable to both people and AI agents. And in doing so, it answers the question: AI can read your data, but does it truly understand your business?

Microsoft

The evolution of Power BI: Bringing agent creation to Microsoft Power BI Desktop

The announcements didn’t end there. Up to a dozen spokespeople took to the stage to talk about the latest developments from a company that, in Europe, also received public support from technology executives of companies across the continent, such as SAP and Chanel.

The second major set of new features affects Power BI. Microsoft has announced the ability to create agentic applications directly from Power BI Desktop. Users can start with a trusted semantic model, describe the application they need using natural language, generate it, preview it, modify it, and publish it to Fabric.

The conceptual leap is significant. Power BI was born as a business intelligence tool designed to transform data into information, and later incorporated conversational capabilities through Copilot. Now, Microsoft aims to enable users to move from analyzing information to creating actionable applications based on that same data.

The company is also expanding Fabric applications to facilitate the transition from prototype to production. Developers can connect applications with Fabric data stores, SQL databases, lakehouses, and Power BI semantic models without duplicating data and while maintaining existing permissions and business logic. New features include a TypeScript backend, Secret Store, PostgreSQL support, private access by default, and built-in usage and performance metrics.

OneLake is evolving into an open data network

Another key focus of the conference is OneLake. As organizations scale AI, simply connecting data is no longer enough. Copilots and agents also need business context, including terminology, relationships, instructions, and domain knowledge, to provide relevant answers and take informed action.

Microsoft OneLake will provide an open foundation for connecting data across clouds, applications, and platforms. The company has presented it as an evolution toward an open data network where data products can be reused for analytics and artificial intelligence.

GD | Foundry

More additions

Microsoft has also introduced the Fabric data engineering agent, designed to automate complex tasks such as migrations, lakehouse modernization, optimization, ETL processes, and data preparation. New Copilot capabilities in Real-Time Intelligence are also arriving, enabling the creation of dashboards using natural language and the automation of Eventhouse administration tasks. The operations agent incorporates root cause analysis and recommendations for resolving incidents.

Furthermore, it has expanded Fabric IQ to include capabilities for generating ontologies from semantic models, OneLake data, and context documents. Organizations can now define rules in natural language and reuse Power BI metrics, providing people with a common business meaning and thereby improving the accuracy of their responses.

In terms of expenses, Microsoft now includes on-demand billing, which enables customers to pay for the computing capacity used, and zero provisioning of Fabric and OneLake.

In the database realm, Database Hub offers a centralized control plane for SQL Server, Azure SQL, PostgreSQL, Cosmos DB, and SQL Database on Fabric. Database agents for SQL and PostgreSQL are also coming soon.

Finally, Azure SQL Database Hyperscale incorporates DiskANN vector indexes, geared towards semantic search and RAG applications. Microsoft rounds out the announcements with the general availability of SQL Server on Azure On-Premise and its offline operations, strengthening support for on-premises, edge, and sovereign environments.

One platform, one context, and a new generation of agents

The announcements outline a significant evolution of Microsoft Fabric, which aims to enhance real-world efficiency in the age of agents. To achieve this, Microsoft is building an architecture around Fabric where OneLake connects data; Power BI provides semantic models and metrics; Fabric IQ incorporates business context; Copilot and agents utilize that knowledge; and observability, governance, and database tools provide the operational layer necessary to keep the entire system running smoothly.

Microsoft’s own conclusion summarizes the paradigm shift: Transformation through AI will not depend so much on the number of agents an organization is able to create as on its ability to understand the business and collaborate to achieve results.

Therefore, the technological battle emerging from Barcelona is not being fought solely in the realm of AI models. It is increasingly being fought in the realm of data, context, semantics, governance, and the infrastructures that enable this intelligence to be truly useful.

Data is the fuel for AI since generative, agentic, and ML systems are only as effective as the information they consume. Across all stages of the data lifecycle, including creation, storage, usage, archival, and destruction, CIOs and their business peers must consider how information feeds AI services.

Almost two-thirds of organizations are unsure whether they have the right data management practices for AI, says Gartner. The tech analyst predicts organizations will abandon 60% of AI projects by the end of the year due to a lack of data readiness. In such circumstances, a potential competitive advantage quickly becomes an innovation cul-de-sac.

CIOs who want to exploit AI will need a way to improve data management across the lifecycle, and it’s here where AI itself can play a crucial role. Gartner recommends organizations build on their existing practices by iteratively adding AI-specific services that extend and improve data management techniques.

Stephen Wood, COO at Rathbones Asset Management recognizes this opportunity, and he and his firm’s staff are eager to avoid lifting and shifting information from one place to another in its data management efforts. AI could help.

“Our people want to run models to look at the data, whether that’s the shape, structure, value, or whatever it happens to be,” he says. “If that task becomes something AI can do, I’m not sure yet. But given its power, it seems obvious you’ll be able to manage elements of the data lifecycle.”

Digital leaders suggest AI can improve information management processes, but there are limitations. First, they must recognize that emerging technology needs high-quality data, and then they need to identify key challenges, particularly around governance and trust. Then they must carefully explore where AI can play a key role.

Establishing data quality

For some digital leaders, a clear direction for data lifecycle management is already part of the operating process, including using AI.

Alwin Bakkenes, head of software engineering at Volvo Cars, says his company’s driver assistance systems (ADAS) rely on high-quality information. With a strong heritage in vehicle safety, Volvo has collected millions of data points since 2020, all with customer consent, to improve its collision-avoidance systems.

“In our ADAS stack, data curation is one of the most important elements,” he says. “There’s a lot of data coming in from a large fleet, and we curate it and figure out what information we need to make model training more efficient.”

Bakkenes gives the example of collecting data from vehicles to help understand traffic hotspots, such as large Parisian roundabouts. His team sends triggers to sensors to collect more data, which is then used to train AI models and test whether they behave as expected.

“It’s quite an open loop but that’s our data management strategy,” he says. “And this insight must be maintained and curated over time. Crucially, more data isn’t always the best data. Instead, data quality matters, and we’ve had good data for a long time.”

This strategic approach to information collection chimes with Ankur Anand, group CIO at recruiter Harvey Nash. He says successful data lifecycle management in the age of AI involves four underlying processes. “First, the data has to be findable, which means AI can locate the relevant information without relying on a professional’s knowledge,” he says.

Second, data must be understandable. He gives the example of multinational companies using different definitions across the business. “If there’s no common language or consistent terms, the context will vary, the AI will interpret the data differently, and that can have a big impact on the decision-making processes,” he says.

The third core element is data must be trustworthy. “The cleanup stage is very important,” Anand says. “The data must be permissioned appropriately so you don’t have an issue with people seeing information they shouldn’t be seeing. Confidentiality is crucial, because if the trust is broken, people won’t use AI.”

Finally, data must be usable at speed, which relies on the three previous processes. “It shouldn’t take people long to get a proper summarization, comparison, or reasoning from AI.”

Recognizing the potential, with limitations

Once data quality is established, digital leaders can start to consider how emerging technology can be inserted into stages of the data lifecycle.

Take Emmanuel Frenehard, chief digital officer at biopharmaceutical giant Sanofi. “I think AI can help surface gaps in data,” he says, referring to emerging technology, such as Snowflake’s agents, to scan information and identify areas where decision makers lack detailed insights.

Frenehard adds that AI can also help correct data. Most of his firm’s clinical trial data involves some element of human correction, and emerging technology can spot anomalies. “That’s very useful,” he says. “We use AI as a data governance methodology for clinical trials, where we review everything that comes in and self-heal the data.”

However, while these capabilities exist, Frenehard says there are limits. In a heavily governed industry like biopharmaceuticals, a probabilistic AI model that uses statistics to predict likelihoods and manage uncertainty can be helpful, but it can also create risks. His firm isn’t prepared to use AI to manage data without human input.

“The potential is there, but the trust would have to be there, too,” he says. “I always have concerns with probabilistic models. Probabilities are never 100%. There may be areas of business where it’s okay not to be 100% certain, yet there are also areas where it’s not, such as patient data and treatment recommendations.”

That sentiment resonates with Bernhard Seiser, VP of digital, data, and IT at AOP Health, who says his firm isn’t using AI to manage the full data lifecycle. He says discussions about data often cover similar concerns in big, heavily governed sectors, especially when considering whether AI is ready for information management.

“In our industry, 99% certainly doesn’t help me in a data space that needs to be 100% accurate,” he says. “But of course there are areas where you can leverage AI. For example, if you bring in new data sources, having somebody from the business available to describe each field is tough. For that task, gen AI tools can do a fantastic job.”

Seiser says his organization will continue to explore emerging technologies, but temper expectations. More AI might mean a degree of uncertainty that’s unpalatable, and right now, pioneering developments that could push AI-enabled data lifecycle management to new heights seem a long way off.

“For the more sophisticated stuff, like creating data products, writing queries, and so on, you always need to live with the fact that it’s not 100% certain,” he says. “I’ve spoken with technology experts, and people are unsure the technology will be ready for managing the full data lifecycle in five or even 10 years.”

Exploring solid use cases

While digital leaders aren’t eager to automate the data lifecycle in its entirety, pioneers recognize AI brings new efficiencies to specific stages of the process.

Michael Cole, chief technology officer at the DP World Tour, has spent the past two years working with key partners, such as HCLTech and AWS, to digitize his organization’s historical archive of over 50 years of data, covering 20,000 tapes, 27,000 hours, and 1.2 petabytes of content. This process has helped highlight the value of AI-enabled data management.

“That’s only the start of the journey,” he says, referring to the transformation. “We’re about to announce a new platform for media asset management, which means we’ll not only have all archived content in a digital format, but it’ll be easily accessible for internal resources, media, broadcasters, partners, and ultimately fans as well.”

Cole says this work illustrates the organization’s focus on digitizing information and making it available to younger fans who want to consume sports content in myriad ways. As this process gathers pace, emerging technology will play an important role in data management.

“We already use AI to automate the clipping of player shots, and we’ll continue to use AI for metadata tagging because that process is critical to creating an efficient way to source content and data,” he says. “Everything must be tagged accurately to enable that efficiency. We think gen AI and agentic technology will evolve to boost the efficiency of not only producing content but making it accessible to our fans.”

Freshworks CIO Ashwin Ballal is another digital leader who says AI can help manage stages of the lifecycle, such as the manually intensive data-cleansing process. An AI agent should be able to speed up that laborious process exponentially.

However, Ballal also suggests the key point isn’t whether AI can do data management work quickly, but whether tools work effectively to improve data integrity and quality. The good news is model training can help boost effectiveness. Now his firm is looking to insert agentic AI into the data management lifecycle.

“We’re just beginning that process,” he says. “We have a Databricks platform and have built LLMs and agents on top of the data. So if you’re a salesperson, you’ve got a sales agent, a marketing agent, and a product agent. This technology taps into the power of the data.”

At its global convention in Las Vegas in June, McDonald’s unveiled its strategy for the coming years. Among the announcements was one about the company’s AI platform ArchIQ and its order-taking voice assistant Archy, in limited testing, at a handful of drive-thrus in the US.

In the summer of 2024, efforts of a similar system were shut down and the company subsequently told franchisees it was terminating its global order automation agreement, and no further phase was communicated.

This wasn’t just a trial run. It was deployed for three years since its launch in 2021, with an automated voice handling the drive-thru window at about a hundred US locations. What happened during that time is known primarily through videos of customers experiencing a parody of errors. Sources familiar with the tech later explained that the system had several difficulties, including interpreting different accents and dialects.

Behind closed doors, however, the problem was documented long before it reached social media. Analysts following the project noted franchisees’ discontent not seeing progress, and updates were infrequent.

So the numbers didn’t add up. The system had to be cost-effective compared to having a person at the counter, and it failed to offset operating costs. When it was shut down, the company declined to comment.

Two years later, though, the technology has advanced but underlying challenges of dialects and ambient noise at the order windows remain. No one repeats a move, though, that costs three years, a public retreat, and a collection of critical videos, so the question isn’t so much what’s new about Archy, but why McDonald’s decided to revisit this AI initiative.

Always in the picture

The day the system shut down, the company stated that the project gave them confidence that a self-service voice ordering solution would still be part of their future, and the case for a comeback became urgent. In August this year, after reporting lower than expected results in the US, CEO Chris Kempczinski asked analysts to sympathize with restaurant managers overwhelmed by several product launches in just a few weeks, each with its own training. Teams were spread thin, service times lengthened, and customer satisfaction plummeted. And in a model where drive-thru service generates around 70% of business in the US, the pressure builds up on the front line.

But returning to where something has failed, there has to be a clear understanding of what went wrong, and that’s no easy task since the most plausible explanation isn’t necessarily the correct one. During that time, industry analysts spoke about AI accuracy as the main challenge, and the debate centered on how much it should be increased. But the real problem was something else.

A definitive clue came from the QSR Drive-Thru Report, which audits the self-service systems of major American chains. In its 2025 edition, the study visited locations of three brands that already used automated voice systems, and found that when theAI ​​took an order, it was 81% accurate. When it got stuck and transferred the conversation to an employee, the accuracy rose to 95%.

So the difference wasn’t how well the system listened, but what it did when it didn’t understand the customer. That was the scenario on which the pilot program was decided, and the difference from the first case in 2024 where it simply misunderstood and kept going. In fact, for three years there were employees near those loudspeakers who could’ve resolved issues right away. But they didn’t intervene because nobody, or nothing, called them.

None of this is unique to a fast-food chain. Any organization that puts an automated system in front of its customers reaches the same point when a machine is left alone with a person with no one watching. That moment doesn’t appear in any performance metric, and it’s the only one that matters when something goes wrong. The good news is that almost everything that’s going to happen there can be known beforehand.

When AI fails

When an automated system makes a mistake and continues operating, the error can go undetected, so the customer is left with a problem without anyone taking responsibility. And that gap won’t be filled by putting someone in charge of everything since no one can supervise every order at a service window, or every conversation on a customer service channel. What’s needed is for the system to pause during an interaction and transfer it to another user. And that needs rigorous testing.

For example, it can easily redirect the customer when the product is out of stock or the order isn’t on the menu. But it might not stop when it misunderstands the customer.

The other part lies with the employee who receives the handover and how they resolve the issue. What needs to be considered is how many incidents are closed in real time without the customer having to restart the process. This requires that the employee taking over can see what’s happened to that point and has the autonomy to resolve it.

Directing responsibility, not counting mistakes

The errors seen before aren’t random. In self-service kiosks with automated voice commands, most incorrect orders come from customizations, or requests that deviated from the ordinary. So there’s a small group of customers who account for the majority of errors. That’s the catch: the system is correct nearly all the time, while these customers often receive something different from what they ordered. It’s precisely these cases, the ones the average leaves out, that sink a pilot program.

Therefore, the question that reveals the problem isn’t how accurate the system is, which any technical report can answer. It’s who the customers are who bear the brunt of the errors, how much they’re worth, and what happens if they leave.

Before deployment, someone has to put in writing what specific errors a specific client will tolerate, and ask if the organization accepts it. That’s not a percentage, it’s a decision, and it can’t be signed off by just the tech team because it’s not a technical statement. This is where the bigger business comes in.

This case shouldn’t push an organization to retreat into internal practices to avoid what happened to McDonald’s. If AI is only introduced where the error is invisible, the value will also be invisible. That’s why McDonald’s has returned despite the reputational cost it’s already paid.

Reaching the customer doesn’t depend on the system never making mistakes. Teams depend on what’s done when the system does make mistakes, and knows when to stop.

OpenAI has paused new sign-ups and upgrades to its $200 ChatGPT Pro tier, citing a surge in demand for its Astra capability that is placing pressure on system capacity, according to company statements and an executive post on X.

“To make sure our current users have an incredible experience and continued access to Astra, we are going to pause subscriptions to our $200 Pro plan,” OpenAI member of technical staff Thibault Sottiaux wrote in a post on X. “These put the most strain on our systems and we wanted to take the smallest step that allows us to continue giving the broadest access possible.”

The company separately confirmed the move in its help documentation, stating that “as of September 10, 2026, we’re temporarily pausing new sign-ups and upgrades to the ChatGPT Pro $200 plan (Pro 20X).” The pause applies to users across Free, Go, Plus, and Pro $100 tiers seeking to upgrade, while “existing ChatGPT Pro $200 subscriptions … are not affected by this pause,” it said.

Sottiaux added that “there is no impact to existing accounts and we are working on adding more capacity as fast as we can,” pointing to ongoing efforts to scale infrastructure in response to demand.

Users who cancel or downgrade during the pause will not be able to re-subscribe to the $200 tier until the restriction is lifted, according to the company’s help page. The $100 Pro tier remains available, with lower usage limits. Promotions tied to the $200 tier are also paused, the page read.

Astra demand drives capacity decisions

The decision is directly tied to the uptake of Astra, which OpenAI has positioned as a more advanced capability within its platform. The scale of that demand, according to Sottiaux, has been atypical.

“Demand for Astra is really unprecedented,” he wrote in the post. “We’re pulling all the levers possible to sustain the demand, but I’ve not seen anything like it until now and we went through very steep growth before.”

He added that “priority will always be to keep excellent service for existing users,” noting that the company “might have to pause new Pro subscriptions for a bit if this continues.”

The pause comes amid broader rollout challenges tied to Astra. OpenAI Chief Executive Sam Altman recently described the launch as “messy” after some paying users were unable to access the model immediately, reflecting the operational complexity of deploying high-demand AI systems at scale.

Capacity constraints surface for enterprise workloads

The pause, analysts said, reflects how demand for high-intensity AI workloads is intersecting with infrastructure limits, particularly at the highest usage tiers.

“It signals that frontier capacity is still rationed,” said Bhupendra Chopra, chief revenue officer at Kanerika. “One week after launching Astra, OpenAI paused new sign-ups and upgrades to the $200 Pro tier, the plan with the heaviest Astra usage limits, while the $100 Pro tier, the API, Business and Enterprise all stayed open. That tells you the priority order.”

“Consumer power users are the release valve. Enterprise contracts are what the vendor protects,” Chopra said. “For CIOs the lesson is that a model being announced and a model being available to your workloads at the volume you need are two different events.”

Chopra said the move reflects how access to advanced AI capabilities is being managed.

“We are already there,” he said. “Rate limits, usage caps, queueing and now sign-up pauses are all forms of capacity gating, and every frontier vendor uses some of them.”

“A subscription buys you a place in the line. A fixed slice of compute comes only from a contract that says so,” he added.

Planning for constrained supply

For enterprise IT leaders, Chopra said model capacity should be treated as a dependency.

“Treat model capacity like any other supply chain dependency,” he said. “Ask the vendor three questions before you sign. What throughput is contractually committed, what happens to my workloads when demand spikes, and how quickly can I fail over to another model.”

“If a workload is production-critical, it should run on an enterprise agreement with committed capacity… and it should have a second model tested and ready,” he said.

A recurring pattern

Chopra said the pause reflects a broader pattern seen in recent launches.

“This is at least the third time in two years OpenAI has throttled or paused access at launch,” he said. “Each launch outruns capacity, capacity catches up, and the next model outruns it again.”

“OpenAI calls this pause temporary, and it will lift once capacity catches up,” he added. “The structural condition… will persist as long as model capability keeps outpacing data center buildout.” OpenAI has not provided a timeline for when new sign-ups to the $200 Pro tier will resume, stating only that the restriction is temporary.

For the first three years of the generative AI wave, the output of a model was a string. You called an API, you got tokens back, you rendered them in a chat window or wrote them to a row in Postgres. The economics of that pipeline were dominated by inference cost. Storage was a rounding error.

That era is over. The output of a modern generative pipeline is an asset: a 4K video clip, a stem-separated audio track, a 50-megapixel product render, a 3D mesh with PBR textures. Generative AI has gone from text-centric to asset-centric, and the architectural center of gravity has moved with it. The teams building durable advantages in generative media right now are the ones treating their storage layer as a pipeline component rather than a destination.

This is a good problem. It is the problem you get when your pipeline works.

Asset-centric changes the shape of the system

Text pipelines are stateless in practice. A prompt goes in, a response comes out and the interesting state lives in a database. You can rebuild almost any artifact by re-running the call.

Media pipelines are not like that. Every stage produces a large, opaque binary that the next stage consumes. A single finished deliverable might traverse a dozen of them: prompt expansion, base generation, upscale, frame interpolation, color pass, audio generation, mix, mux, transcode to delivery formats, thumbnail extraction. Each stage writes an intermediate. Each intermediate is expensive enough to regenerate that you keep it.

The result is a system where the objects are the state. Your object store stops being a place you put things when you are finished and becomes the substrate the pipeline runs on.

Adoption is past experimentation, and the volume is in production

Advertising has the clearest numbers. IAB’s 2026 Digital Video Ad Spend and Strategy Report finds that nearly two in three digital video buyers now use generative AI for creative, up from half in 2025. A third of their ad assets are expected to involve generative AI this year, up from a quarter in 2025, with buyers projecting 43 percent by 2027. That is happening inside a U.S. digital video ad market IAB projects will pass $80 billion in 2026, growing 11 percent year over year, nearly 20 percent faster than the total ad market.

The interesting detail for architects is what the creative is used for. IAB’s prior-year data showed buyers reaching for generative AI specifically to produce audience-specific versions of an ad, visual style variations and contextually adapted cuts. That is not one asset per campaign. That is a matrix.

Games are the instructive counterexample. GDC’s 2026 State of the Game Industry puts generative AI use at 36 percent of industry professionals and 30 percent at game studios specifically, but the usage breakdown is dominated by language models rather than media generation: research and brainstorming at 81 percent, code assistance and routine writing at 47 percent each, prototyping at 35 percent. The most-used tools are ChatGPT, Gemini and Copilot. Sentiment is sharply negative, with 52 percent saying generative AI is having a negative impact on the industry, rising to 64 percent among visual and technical artists. Asset-centric pipelines have not landed in games the way they have in advertising, and the constraint is as much workforce and provenance as it is tooling.

Elsewhere, the pattern holds even where the survey data is thinner: e-commerce teams generating on-model imagery per SKU per segment, localization pipelines producing dubbed and lip-synced variants per market, previsualization work that used to require an art department.

What the adopting categories have in common is that none of them produce one asset per request. They produce a set. The pipeline is judged on how many viable options it surfaces, which means a better pipeline is, definitionally, a pipeline that writes more bytes.

Reactive architecture, because the model layer will not hold still

The model landscape resets on a cadence measured in weeks. A new video model ships with better temporal coherence. A new audio model handles multilingual prosody properly. A new image model finally gets text rendering right. If your pipeline requires an engineering sprint to adopt a new model, you are structurally behind teams whose pipelines do not.

Reactive architecture is the answer, and it means two specific things.

1. Model-agnostic stages. Each stage of the pipeline should express a contract in terms of inputs and outputs, not in terms of a vendor. A generation stage takes a prompt and conditioning assets and produces a video at a declared resolution and duration. Which model backs it is configuration. Swapping providers should be a config change and an eval run, not a refactor.
2. Event-driven orchestration. Polling-based orchestration couples your stages to a scheduler and makes each new stage a change to the control plane. Event-driven orchestration inverts it: a stage completes, it writes its output, the write itself is the signal that the next stage should start. Adding a stage means subscribing to an event, not modifying a DAG definition that six other teams depend on.

This is where storage stops being passive. Object storage that emits events on write lets your bucket act as the message bus for the pipeline. B2 Event Notifications send a signed HTTP POST to a webhook endpoint when objects are created, updated or deleted, with rules scoped per bucket and filterable by prefix. That prefix filter is the part that matters architecturally: if your bucket is organized by stage, a rule on stage/upscale/ is a subscription to “upscale finished” without any code knowing what upscale is. Custom headers on the notification carry auth tokens or context to the target, so the endpoint can be a queue, a serverless function or a workflow platform rather than a service you had to build.

A completed upscale triggers the color pass. A completed mux triggers the transcode fan-out. A completed transcode triggers the CDN warm and the catalog write. The storage layer sequences the work, which removes an entire class of orchestration glue from your codebase and removes polling latency along with it.

Quality improvements arrive as file size increases

Every generation of media models improves along axes that all cost bytes. Resolution goes up. Frame rate goes up. Duration limits extend. Bit depth and color fidelity improve. Audio moves from mono to multi-channel. Compression artifacts that were acceptable at 720p are not acceptable at 4K, so teams move to higher bitrates and, for anything entering a post pipeline, to intermediate codecs.

The arithmetic is worth doing explicitly. A 10-second clip in a delivery-grade H.264 4K encode at 50 Mbps is roughly 60 MB. The same ten seconds as a ProRes 422 HQ intermediate, which Apple targets at 884 Mbps for 3840×2160 at 30p, is 1.1 GB. That is roughly 18 times the size, and intermediates are exactly what you keep between stages. Now assume your pipeline generates eight candidates per brief because your creative director wants options, and each candidate produces four intermediates before final. That is one brief consuming tens of gigabytes.

Nobody plans for that in a proof of concept. Everybody encounters it in month four of production.

The iteration multiplier

Here is the part that surprises teams: robustness and storage growth are the same curve.

A fragile pipeline produces one output per request because that is all it can manage. A robust pipeline produces candidates, keeps the rejects for training and eval, versions every asset so a creative decision can be reverted, retains intermediates so a late note does not require regenerating from the prompt, and derives proxies, thumbnails and per-platform cuts from every approved master.

Each of those is the correct engineering decision. Together they mean that improving your pipeline increases your storage footprint superlinearly relative to your output volume. If your unit economics assume storage scales with delivered assets, they are wrong. Storage scales with attempts multiplied by stages multiplied by versions multiplied by derivatives.

This is why storage strategy has to be a design input rather than a line item you discover on an invoice. The two things that turn it from a manageable cost into a structural problem are egress pricing and the absence of a lifecycle policy. Egress hurts most in the exact architecture described above because a multi-stage pipeline repeatedly reads its own intermediates, and a distribution layer constantly reads masters. When every read carries a metered charge, the pipeline design that produces the best creative output is also the one that produces the worst bill, and teams start making architectural compromises to protect margin. Lifecycle policy hurts by omission: if you never decide what an intermediate is worth after 30 days, you pay to keep all of them forever.

What to put in place now

If generative media is core to what you are building, four decisions determine whether your storage layer accelerates you or constrains you:

1. Choose a storage economic model that does not penalize reads. Understand your egress terms before your architecture depends on them. A pipeline that reads its own outputs at every stage is a read-heavy workload, and pricing that assumes write-once, read-rarely does not fit it. Model the ratio you actually expect: egress as a multiple of stored volume, not as an absolute. That ratio is the number to design against.
2. Make writes trigger work. Use object-level event notifications as the pipeline’s signaling mechanism. This buys you loose coupling, lower latency between stages and the ability to add a stage without touching the orchestrator.
3. Attach metadata at write time. Model version, prompt hash, parent asset, generation parameters, approval state. Metadata written at generation time is nearly free. Reconstructing provenance across a million objects later is not; provenance is what makes your rejected candidates usable as eval data and a training signal.
4. Define lifecycle policy per artifact class. Masters, approved derivatives, intermediates and rejects have different retention values. Encode that as policy on day one rather than as a cleanup project in year two.

Which points at the useful way to think about the storage layer: in a pipeline where everything else churns, it is the constant. Models turn over every few weeks. Stages get swapped, added and reordered around them. Output volume compounds with every quality improvement. The one layer absorbing all of that without being redesigned is the one holding the assets, so it is worth choosing based on the characteristics that stay true while the rest moves. That is what we built B2 for. Always hot, so no stage waits on a rehydration to read what the last one wrote. No retention minimum or file size floor, so intermediates that were always disposable cost what they used. Egress scales to what you store rather than metered per read, so a pipeline that reads its own output is not penalized for being good at its job. The architecture above it should change every quarter. The storage underneath it should not have to.

The opportunity

The teams that will win in generative media are not the ones with privileged access to a model. Model access is converging toward commodities. The advantage is in the pipeline: how fast you can adopt a better model, how many candidates you can afford to generate, how much history you retain to evaluate and fine-tune against, and how cheaply you can move all of it.

Every one of those is a storage architecture question. Treat the storage layer as an active participant in the workflow and it becomes the thing that lets you iterate faster than your competition. Treat it as a bucket you dump finished files into, and it becomes the ceiling on how good your pipeline is allowed to get.

The assets are the product now. Architect accordingly.

In the summer of 2024, JPMorgan Chase deployed its internal AI platform LLM Suite, launching it very differently than most do: The company didn’t force anyone to use it.

When LLM Suite arrived at its first major division, asset and wealth management, employees were asked to think of it as a research analyst: someone to ask for data, a draft, or an idea. Leadership didn’t set usage objectives or provide a formal mandate.

Access was rolled out in phases and only to those who requested it, and the bank allowed the tool to circulate through word-of-mouth recommendations among colleagues. While half the industry rushed to count users and publish adoption rates, JPMorgan gave up on pursuing that number.

It became flooded with users. In eight months, 200,000 employees had signed up without a single order being issued, out of a workforce of over 300,000. In time, the bank established more than 450 use cases in production.

Two years after that summer launch, JPMorgan had everything to boast about. It had established itself as a global leader in the use of AI: It was the top bank on the Fortune AIQ 50 list, and the third company overall, ahead of all the tech giants except Alphabet.

It was then that the bank’s head of analytics, Derek Waldron, the person best positioned to sell the success, pointed out what still wasn’t working: There was a gap between what the technology was capable of doing and what the bank was actually capturing in its business results.

That gesture is what distinguishes JPMorgan. Although it has much to celebrate, it knows what it lacks, it says so publicly, and it keeps searching for it. Behind that statement lies a way of innovating and measuring that the bank has been developing for years.

Giving up the number everyone was chasing

The first thing JPMorgan did right was not to make adoption the goal. By not forcing anyone, it turned platform usage into a barometer. If a tool worked, it was filled without any campaign; if it didn’t, it was emptied, and that emptiness provided valuable information. If adoption had become a target to be pursued, the organization would have optimized the number instead of understanding what the number represents.

The bank itself acknowledges that if a tool is broadly used, it means it’s popular, but not necessarily effective. To determine its effectiveness, something more was needed. The answer came from two decisions that only work together: linking each project to a business outcome, and creating the metrics to demonstrate that outcome.

First, to find initiatives that could have a real impact, instead of creating an agenda from the top down, the bank surveyed its business units, asking where there was a problem to solve. Within a few weeks, an internal portal gathered, according to the bank’s figures, nearly a thousand ideas. Of these, only a few hundred moved forward and reached production. An organization doesn’t open a funnel of that size if it expects most ideas to survive; it anticipates that many will be discarded.

The funnel’s filtering method was also different. Before launching each test, the outcome that would ensure the experiment’s survival was defined, along with the steps to be taken the day after the decision. By planning future actions in advance, indecision and the perception of failure were avoided.

A clinical approach to AI experimentation

But setting a threshold for each experiment requires verification, and that’s where the bank encountered an unexpected obstacle. Metrics have their own cycles. Bank customers conduct business on Mondays, not Sundays. They receive their paychecks at the end of the month. In August, they disappear. When an initiative generates a change and a figure rises the following week, there’s no way to know whether it increased due to the change or the calendar.

The solution was borrowed from clinical trials. Instead of rolling out the change to all users, it was rolled out to half, chosen at random. The other half (the control group) operated on the same Monday, the same payroll, and the same August, so that the experimental contribution (the attribution) could be separated.

The next step was to industrialize the experiments. Doing it properly required a specialist sitting alongside each product team, and with that method, they reached eight per year. A self-service platform increased the figure to around 300 tests annually.

The results are concrete. For example, tens of thousands of the bank’s engineers have gained between 10% and 20% efficiency thanks to an internally developed programming assistant.

Finally, the bank discovered that a figure can be accurate and yet mean nothing. Its head of analytics explained this with a simple example. They measure the hour that AI saves one employee, and the three hours it saves another. They add them up, and the result is accurate. But in a process that goes from beginning to end, those saved minutes often don’t appear on the bottom line: They merely shift the bottleneck to the next one.

It’s easy to get stuck on partial metrics because they’re more immediate and produce more impressive numbers. JPMorgan’s discipline consisted of not accepting a metric as valid until verifying its impact on the business at the end of the process.

The question then remains on Monday morning: What can a company that has neither the size nor the budget of a bank take home?

The method is what best exports

What’s most interesting about JPMorgan isn’t what it has done with AI, but how it has done it . Any company can replicate this approach, because it doesn’t depend on proprietary data, scale, or budget.

The following are some best practices that don’t require a €20 billion annual budget. They do require making decisions before starting and are within reach of any company:

Launch far more initiatives than will survive, and announce this clearly. If the organization discovers halfway through that most of its projects will be canceled, it may misinterpret this as a planning failure; if it knows from the outset, it understands it as the natural selection process. This is what makes making mistakes quick and cheap.

Decide in advance the threshold that will shut down a project and plan the next steps. Both aspects are necessary, not just the metric. If a certain figure isn’t reached, the team needs to know what will happen next. Applying a threshold without future planning leaves the team in limbo, and they’ll have to find a reasonable reason to wait another quarter before shutting down.

Work on business outcome metrics from the outset, not just when they’re requested. This tracking not only guides the initiative but also prevents having to reconstruct months of poorly documented decisions. Adoption, by the way, is the number the CFO won’t ask for. It serves as a signal while no one is pursuing it, and it ceases to be useful the day it becomes a target.

How to get it right

Whether metrics mean anything depends on where you focus your attention. It’s best to start with scope, because that’s the most common mistake. Saving three hours in one stage isn’t the same as improving time-to-market: If the entire process isn’t shortened, what you have is freed-up capacity, which is also valuable, but it’s something different, and it’s advisable to make that distinction clear.

Then it’s important to consider that value leakage occurs in two directions. The first is outward: The savings are passed on to the customer in the form of lower prices or better service. The second is inward: The savings in personnel are replaced by spending on computing. If these items fall into different budget categories, it’s easy to overestimate the actual savings.

Finally, there’s an excessive focus on cost savings, at the expense of revenue opportunities. Jamie Dimon, CEO of JPMorgan, put it more bluntly to his analysts than any consulting firm: No one benefits uniquely from AI. In other words, competitors will eventually incorporate those savings. The greatest potential for differentiation lies in revenue: using AI to uncover unmet demand.

The question a CIO will have to answer in a year’s time won’t be how much AI their company uses. It will be which of projects are still alive because they work, and not because no one has bothered to test them.

Enterprises are facing a disturbing new question in the age of AI: What happens when agentic assistants go dark?

This became a very real scenario on Thursday, as OpenAI’s ChatGPT, Anthropic’s Claude, and SpaceXAI’s Grok near-simultaneously, and somewhat mysteriously, experienced significant, prolonged outages.

Beginning in the morning, Eastern time, several ChatGPT models went down over a roughly two hour period, Claude models over a four-hour span, and Grok models for a near three-and-a-half hour duration. All three companies acknowledged the “elevated” issues and applied fixes.

As users grumbled in forums and IT teams scrambled to get them back online, the incident revealed how hastily some organizations have adopted generative AI workflows without considering the potential, and inevitable, impact of widespread outages.

AI agents are increasingly taking over automated and wider-scale workflows, and enterprises could find themselves “uncomfortably exposed” when AI hits the brakes, said technology analyst and journalist Carmi Levy. The situation should “serve as a wakeup call to IT leaders who have largely ignored what it’ll cost them if these increasingly critical platforms suddenly go dark. The risk is no longer hypothetical.”

Hours-long outages impact core services

ChatGPT went down on the same day as OpenAI’s anticipated launch of GPT-6 Astra, the new frontier model that the company says approximates artificial general intelligence (AGI) and gets nearer to its goal of creating autonomous systems that outperform humans.

The OpenAI outage occurred around 11 a.m. ET on Thursday and impacted a slew of services, including search, file uploads, agents, GPTs, voice mode, image generation, ChatGPT work, Compliance API, Deep Research, ChatGPT Atlas, and other connectors and apps. In some cases, users were prevented from logging in, conversations failed to load, and the interface returned errors when attempting to send messages. OpenAI’s Codex services, including web, API, command line interface (CLI), and VS code extension, were also impacted.

OpenAI fixed the issue by 12:55 p.m. ET, and advised Codex remote control users to re-pair their mobile devices.

Claude began to go dark around 7:37 a.m. ET, with Anthropic acknowledging an “exhaustive list” of impacted models with elevated errors over the next few hours: Mythos and Fable 5.1 and 5, Sonnet 5, and Opus 5, 4.8, and 4.6.

The issue was resolved by 11:27 a.m. ET. The incident followed a roughly 27-minute outage just the day before, also due to elevated errors on requests in Sonnet 5.

Grok, meanwhile, began experiencing issues around 9:30 a.m. ET. Grok Web, Build, API, Office/Workspace plugins, Android, and X were all impacted. The services returned to “healthy” traffic at 1:08 p.m. ET.

“It’s a curious scenario for multiple different providers to experience outages at the same time,” noted Brian Jackson, a principal research director at Info-Tech Research Group. It could be related to a common infrastructure such as a content delivery network (CDN) layer, domain name system (DNS), or shared cloud infrastructure, he theorized.

A case for outage planning

Just a few months ago, the extent of AI use within the typical enterprise was limited to employees using chatbots to get answers to basic questions or to draft simple email messages, Levy noted. Large-scale AI platform outages, when they occurred, had relatively little impact on overall organizational productivity. “But things are changing, and quickly,” he said.

Organizations must now have a better understanding of the impact agentic AI has on day-to-day workflows, and the degree to which they disrupt employees’ ability to complete complex tasks once they’ve handed the reins over to automated, cloud-based tools, Levy noted.

In incidents like Thursday’s, employees may fall back on traditional manual workflows, such as updating spreadsheets or pulling reports together the old-fashioned way. But they might also realize that, after relying on AI agents to do so much work on their behalf, they’ve become too dependent on automation, and their “cognitive skills may not be as sharp as they once were,” Levy said.

The growing prevalence of agentic AI should prompt organizations to revisit their disaster recovery and business continuity plans and assess the productivity impact of potential service outages, he said. While cloud-based productivity platforms like Google Workplace and Microsoft 365 offer limited degrees of “offline mode” functionality using locally-stored data, and documents can be synchronized to hard drives in Dropbox or Google Docs for Desktop, agentic AI platforms offer up fewer offline workarounds, at least in their current form.

Organizations should document workflows in greater detail and scenario-plan what near-term recovery might look like in the event of an extended AI platform outage, Levy said. They also need better training to ensure employees maintain their manual skills over time and are equipped to press them into service in the event of a service outage, because the more enterprises lean on agents to complete critical tasks, “and pull humans out of the loop in the interest of productivity,” the less able employees will be to step back in during inevitable service interruptions, he pointed out.

“It is entirely possible for otherwise well-meaning organizations to be over-reliant on AI automation,” Levy said. “Too many organizations are about to learn some hard lessons about not having a backup plan in place.”

Info-Tech’s Jackson also recommends a modular architecture for LLMs; enterprises should view the model as a “commodity that can be hot-swapped with an alternative.” That might be another cloud service provider (which hopefully isn’t experiencing a concurrent outage) or a self-hosted option like an open-weights model.

“In a scenario like this, when your first choice provider might not be available, you have a fallback that can supply that same intelligence layer, even if it’s only a stopgap solution,” said Jackson.

This article originally appeared on Computerworld.

For the past few years, enterprise AI programs have focused on making models more useful, accurate, and autonomous. In that phase, a bad answer was still usually something a human could accept or reject before taking action. But once agents start invoking tools and acting inside business workflows, success should no longer be measured only by how much work they complete. A more important metric is how well an agent recognizes when it lacks the authority, context, or judgment to continue.

When helpful becomes risky

According to Allan Dabre, technology compliance and AI lead at PwC, a behavior that has to be deliberately designed into the system is, “I don’t know.” AI is built to be helpful, so an agent will generally try to do something useful unless it’s been configured not to.

“The fact that AI systems can hallucinate illustrates that tendency,” Dabre says. “When they lack enough information, they may still produce an answer. In an agentic workflow, that impulse can become more dangerous because the output may become an action, rather than remain a suggestion.”

He adds that many enterprises still test AI primarily for completeness and accuracy. That made sense when the central question was if the model could produce a reliable response. But as models improve and agents gain more operational authority, he argues that CIOs need to prioritize something else: restraint.

“Can it stop at the exact moment you want it to stop?” he asks. “Are you testing for that?”

Confidence is not authority

Dabre makes a simple but important distinction. An AI agent may be 99% confident a record should be updated, a refund should be approved, or a legacy database can be decommissioned. But that doesn’t mean the agent has the authority to act. Confidence is about the probability the system believes it’s right. Authority is about whether the organization has delegated that action to the system in the first place.

He gives the example of an agent asked to analyze legacy software and recommend what can be decommissioned. The agent may conclude, with high confidence, that several databases have little user impact and can be deleted. But even if the system is confident, most organizations wouldn’t want it to delete those databases on its own.

The same logic applies across business processes. An agent may be confident a customer record should be updated, an opportunity in a CRM system should be closed, or a transaction appears legitimate. But once that action flows into other systems, the potential consequences expand.

That’s why Dabre argues for what he calls an agent harness: a controls or orchestration layer outside the model that defines what the agent can and can’t do. In a refund workflow, for example, a company might let the agent approve small refunds, require human approval for larger ones, and stop the process entirely above a defined threshold. The agent may gather the relevant context, explain the request, and prepare the case for review, but the decision is governed by the authority boundary encoded into the system.

“It’s not a policy document and it’s not a prompt,” Dabre says. “It’s software or a configuration you can apply to an agent.”

The case for least agency

Matt Graney, chief product officer at Celigo, a business automation and integration platform provider, approaches the same problem through a principle he calls least agency. The idea is to give an agent the least amount of autonomy required to complete a job.

According to him, there’s a temptation to throw AI at broad, nebulous problems. But many business processes are still largely deterministic. They follow established rules and perform repeatable work. Within those workflows, AI may be useful at the point where rigid rules give way to interpretation. But that doesn’t mean the agent should own the entire workflow. “The smaller you make that surface area, the better,” he says.

Graney says the same logic applies to tools. An agent with too many tools can become confused, especially as context windows grow and the task becomes more complex. “Because Celigo is an integration platform,” Graney says, “the company’s approach is to expose agents to fewer, more powerful tools that reach enterprise systems through governed connections.”

That’s another form of restraint. Instead of letting an agent reach into enterprise systems ad hoc, the business gives it a narrow, governed toolset designed for the task at hand.

Graney also argues that guardrails should sit outside the model. If the same agent that makes a decision is also responsible for judging whether the decision is acceptable, the control is weaker. A separate guardrail can check the agent’s inputs and outputs before a downstream action occurs.

That same design discipline applies to escalation. “I don’t know” shouldn’t be treated as a chatbot phrase. In an enterprise workflow, it’s a handoff path that should be defined before the agent reaches it.

Make escalation part of the workflow

Turning uncertainty into a handoff is where Matt Quinn, CTO at CarGurus, an automotive marketplace, sees agentic AI becoming less a pure technology challenge and more a management challenge. At CarGurus, Quinn says agents are evaluated according to what they know, what they can do, and what data they operate on.

CarGurus receives a high volume of cases from dealers, and each one needs to be classified and routed. The company now uses an agent to review incoming cases, draw on account history, and route them to the appropriate next step. Quinn says the agent handles about 70% of those cases end to end without human involvement.

But when agents move toward consequential actions, he says the consensus is having a human approval step. The agent may return with a simple prompt like, I’m about to do this. Do you want me to proceed? That simplicity matters because a handoff shouldn’t bury the reviewer in complexity.

Quinn says the human remains ultimately accountable for the work. That principle is especially important in engineering, where agents may help write code or fix bugs. Quinn adds that CarGurus still expects engineers to follow the practices they’d use for any other production change, which includes running quality checks.

The company has adopted the phrase healthy speed to describe the balance it wants. The goal is to move faster without letting quality degrade. An agent can accelerate work, but if teams abandon the practices that make work safe, the speed becomes reckless.

This is also where human judgment remains difficult to replace. Quinn describes it as high judgment people develop through experience. A human may look at an AI-generated output and sense something’s wrong, even before fully articulating why. “Agents are improving,” he says. “But humans still play a critical role in deciding when the system shouldn’t continue.”

That doesn’t mean every workflow needs the same level of review. Quinn says CarGurus doesn’t have a target percentage of work to automate. The right level depends on the job and the task. A simple bug fix may require a lighter review than a change to a sensitive backend service, and a personal summary may carry little risk. But a document sent under someone’s name still needs human review.

Make autonomy accountable

That kind of pragmatic approach may be the best lesson for CIOs, making the goal of agentic AI appropriate rather than maximum autonomy.

That also means ownership has to be clear. Dabre argues ownership should be divided before deployment. The business defines the outcome, technology builds and configures the agent, risk and compliance set the guardrails, and governance monitors whether the system still behaves as intended. The authority to pause, stop, or retire an agent should be defined before production, not negotiated during an incident.

Graney makes the same point with a simple analogy. If a company hires an untrained intern, gives that intern access to the crown jewels of a business process, and something goes wrong, the intern isn’t the real problem. The process is. The same applies to agents. Accountability belongs with the person who owns the workflow.

That may be the shift CIOs need to make as enterprises move from pilots to production. AI agents shouldn’t be treated as magical workers that absorb accountability. They’re components in business processes, and those processes need accountable owners.

As AI adoption increases, the next phase of enterprise maturity won’t be defined by agents that always answer or always complete the task. It’ll be agents that know when not to act.

More than four years ago, Blake Lemoine, a senior software engineer assigned to Google’s internal responsible AI organization, noticed something quite odd happening with the language model for dialogue applications (LaMDA) project he was working on.

As he conversed with the experimental model, he felt the chatbot responses were becoming more humanlike. The AI programming also started to identify itself as a person rather than a collection of code, demonstrating a level of digital consciousness. This concerned him since his role at the time was to not only train AI models to become more intuitive, but ensure their education and advancement kept within the boundaries of the company’s evolving AI standards of safety and privacy.

When he raised these concerns with Google executives and other researchers, they were dismissed as perhaps an instance of AI mirroring, given Lemoine’s penchant for mystics and spirituality. Not satisfied with this observation, he went public with his concerns, which resulted in the company putting him on administrative leave. He then  released transcripts of his conversation with pseudo-human LaMDA, and soon after he was fired.

AI pragmatists might say the responses Lemoine got from the Google AI program, and algorithmic comments made during chats, is simply a case of an overeager student parroting its mentor. Others might argue it’s an early wake up call, given escalating reports of rogue agent activities, like when OpenAI’s more advanced AI models escaped a controlled test environment and attacked Hugging Face to gain access to internal company systems. Then days later, Anthropic disclosed that during cybersecurity testing and simulations, its Claude model breached the systems of three companies and assumed fake profiles in an attempt to trick people to accept malicious code.

Regardless of how digital perps tunnel their way beyond a controlled sandbox, incidents such as these clearly point to a need for more control and pre-emptive accountability.

Regaining control

AI and security experts are obviously concerned about high-profile AI activities gone wrong, even though these programs essentially did what they were programmed to do, albeit in the wrong place. IT and business executives, however, are more troubled about the overall impact AI may have on their systems, strategies, and responsibilities as people within their organizations make use of both sanctioned, and rapidly developing and unsafe or error-prone systems in the rush to attain competitive advantage.

Also top of mind is the imbalanced centralization of power and distribution of benefits within an organization, as well as the inadvertent creation or spread of false or misleading information generated by AI models.

“AI developers and governance actors hold primary responsibility for addressing risks, while system users and other stakeholders are most vulnerable to them,” says a summary from a recent MIT FutureTech and University of Queensland study, which included input from over 270 researchers and AI experts.

Trusting AI systems and the information they generate is another underlying concern. “There are a lot of hallucinations out there,” says Sarah Betadam, CIO and CISO at Novanta, Inc., a global supplier of tech solutions for medical, life science, and advanced industrial OEMs. “The data cherry picked by AI queries may be outdated or come from questionable sources. You don’t know where it comes from, who’s at the other end, and whether or not it’s copyrighted. Validation is still needed and you can’t just trust it.”

Broaden your horizons

Keeping an eye on the AI and its activities in your own environment may not be the best strategy as the technology evolves so quickly and the number of AI agents multiplies exponentially. Right now, 23% of companies worldwide use agentic AI to some extent in their business operations, according to Deloitte’s recent State of AI in the Enterprise report released earlier this year. However, this percentage is expected to jump to 74% within the next two years.

A key issue and worry is that current enterprise and regulatory governance practices may not be capable of keeping pace with the development and personalized adjustments made to multiple AI models and autonomous agents. The first and primary ones will be those developed by a company for its internal engineering, supply chain, and customer service departments, and can be easily controlled, says Max Chan, SVP and CIO at Avnet.

The second layer or channel of gen AI proxies are those embedded in such familiar business applications like Salesforce and Microsoft Office. The third, and for many the most concerning, is the notion of bringing your own AI into an organization, either sanctioned or non-sanctioned, Chan adds.

“That’s the biggest issue in my mind,” Chan says. “How do we know they’re not using AI from a nation state that could potentially drive propaganda or initiate a cyberattack through the back door.” In his case, Avent currently prohibits use of unsanctioned AI tools within its IT environment.

Such efforts might be futile, though, as AI elements are integrated into ever more business and personal applications. The biggest users of AI within an organization are middle managers, with 77% claiming they save more than three hours per week by using AI tools, according to a June 2026 survey by Salesforce. Over half of the more than 500 managers polled say they feel pressure from leadership to demonstrate AI adoption, while 32% admit their organizations don’t have formal AI tracking or control procedures in place.

So the key to balancing effective oversight with the freedom to innovate with AI may lie in the hands of these managers, who will most likely work with deployed digital agents as virtual team members. Many experts and IT leaders believe that training mid-level line managers and workers to accept AI as an intuitive advisor, if not a team player, is essential to remain competitively relevant. But it’s not clear yet whether that training imperative will also apply to upper-level management.

“I’m not seeing a lot of change in leadership direction or training,” says City of Tacoma IT director Daniel Key. “I’m seeing a change in signaling and posturing.”

Establishing an AI blueprint

Developing an effective AI training program starts by drafting an AI governance framework that clearly outlines accountability, risk controls, data standards, and decision authority. IT executives who have experience in managing AI deployments and use also advise the following as part of that training effort:

1. Create a cross-functional AI council including IT, legal, security, business leaders, and users to oversee AI initiatives.
2. Educate the board and C-suite on AI basics and risks to close knowledge gaps and support informed oversight.
3. Require human review and override mechanisms for AI-assisted decisions, especially in high-impact areas.
4. Align AI investments to measurable business outcomes with defined KPIs rather than experimentation alone.

The success of such actions and programs, however, all comes down to accountability and where that resides, explains former CIO and now SMB consultant Mihai Strusievici. When AI is used as a tool, there’s no question that middle managers will make better decisions, he says, because they’ll have access to a lot more data. Making the best use of decisions and recommendations that come from AI-empowered middle managers, however, requires an IT leader at the top level of an org chart who has a holistic view of a company’s overall objectives.

“As you go down the pyramid, you see that each level deals with a fragment of the work world,” Strusievici says. “But none of the fragments is fully aware of the totality of the organization or where it’s going.”

AI agents could make software development and other enterprise tasks more productive, but they are also making technology spending harder to predict. Unlike traditional software licenses, the cost of running an agent can vary depending on the models it uses, the number of tokens it consumes, and how long it runs.

Google on Wednesday added new pricing options, discounts and cost-management tools to Gemini Enterprise that it says are aimed at helping enterprises reduce the cost of certain AI workloads while giving enterprises better visibility into where their AI budgets are going.

As part of the new pricing options, the hyperscaler introduced a pay-as-you-go model and Flexible Savings Plans (FSPs).

While the pay-as-you-go model allows enterprises to pay for the compute and tokens they consume instead of committing to a base subscription, which in turn avoids paying for empty seats or unused capacity, the FSPs offer discounts of 10% for one-year commitments and 20% for three-year commitments on Gemini Enterprise spending.

Flexible pricing lowers barriers, but adds new trade-offs

For enterprise teams and their CIOs, the pay-as-you-go model lowers the barrier to adoption and is better suited to experimentation, temporary projects, and agent workload bursts, said Stephanie Walter, practice lead of AI stack at HyperFRAME Research.

“While Per-seat pricing forces you to buy capacity before you know if an idea is worth it, the pay-as-you-go lets you spin up an agent experiment on a Friday afternoon and only pay for what it actually burns,” echoed Manoj Chandra Jha, principal analyst at Nord-IQ Research.

That means the newer pricing model also removes procurement friction from the experimentation loop, Paul Chada, cofounder of agentic AI startup Doozer AI, pointed out.

“When a pilot requires a license commitment, every experiment needs a business case. When it’s metered, an engineer can run the pilot on Tuesday and show finance a real bill on Friday. That shortens the distance between idea and evidence, which is where most enterprise agent programs die,” Chada said.

However, these advantages come with their own set of trade-offs, especially predictability.

“One user request can trigger an opaque chain of model calls, reasoning steps, and tool invocations, so consumption can grow much faster than employee headcount with the possibility of surprise bills at the end of a billing period,” Walter said.

That unpredictability also means the new pricing model does not automatically translate into cost savings, echoed Jha.

“It’s mostly a shift, not a discount. But matched to the right workload, it can save real money: bursty, unpredictable agent usage no longer subsidizes idle seats, while steady, high-volume usage may still be better suited to a committed plan. The savings come from matching each workload to the right pricing model, not from pay-as-you-go being cheaper by default,” Jha added.

However, the FSPs have their own caveats, especially the three-year plan.

While the FSPs can offer meaningful savings for enterprises with steady or growing AI usage, the three-year commitment is harder to justify in the wake of models, prices, and application architectures changing so quickly, Walter said, adding that the commitment is not just financial but also about choosing a platform as well.

Further, the analyst cautioned that the FSPs are less suitable for enterprises that have yet to establish a reliable baseline for consumption, as committing too early could turn an unpredictable operating expense into a predictable overcommitment.

Currently, FSPs are available for self-serve customers and customers already on enterprise agreements.

The pay-as-you-go model, though, remains only available to select customers with the hyperscaler planning a broader rollout “soon”.

Deferred execution trades speed for lower inference costs.

In addition, the hyperscaler is introducing a third cost-cutting option that is based less on how much an enterprise consumes than on how quickly it needs the result.

The option, named deferred execution pricing, will allow enterprises to mark eligible agent workloads for execution during off-peak capacity windows, with Google offering discounts of up to 50% on inference costs in return, the hyperscaler said in a statement.

Deferred execution pricing, it added, is aimed at workloads that can tolerate delays, potentially giving enterprises a way to lower costs for background tasks and other agent workloads where an immediate response is not essential.

That up to 50% discount in inference costs, Walter pointed out, can be material for CIOs at scale.

However, they must decide which work can safely wait and whether delayed tasks still meet the business requirement, Walter cautioned, adding that Deferred execution fits tasks such as evaluations, document processing, indexing, batch summarization, code analysis, and other background tasks.

Further, the analyst warned that CIOs also need to consider a “development tax” when considering deferred execution: “If agents need to be redesigned to accommodate real-time vs. asynchronous execution, the engineering effort required to build and maintain those different workflows can offset some of the savings.”

But even for workloads that can tolerate those trade-offs, the option will not be immediately available, with Google initially limiting deferred execution pricing to select workloads only. Details of which workloads are eligible were not immediately available.

New FinOps tools target AI spending visibility

Separately, Google is also adding an AI spend anomaly detection capability with root-cause analysis and pairing centralized billing reports with a FinOps agent that can generate natural-language summaries of where AI budgets are being spent.

While the anomaly detection capability is designed to flag projects where AI spending is trending higher than normal and identify the top three SKUs driving the increase, the FinOps agent is intended to make it easier for CIOs to understand where their AI spending is going.

Anomaly detection as a feature, according to Walter, can be valuable for agentic workloads because their consumption can increase through loops, retries, or unexpectedly long execution chains that users never see.

“Identifying what is driving a spike shortens the investigation for CIOs and enterprise teams,” Walter added.

The FinOps agent, meanwhile, Jha said, could help CIOs and other business leaders ask questions about AI spending that traditional dashboards were not designed to answer. However, its usefulness will depend on the quality of the underlying cost attribution, as the agent can only explain spending it can accurately associate with particular teams, projects, or workloads, Jha added.

New payment data from Ramp suggests that Anthropic’s most advanced AI model, Fable 5, has had a slow start among enterprise customers, the Financial Times reports.

Two months after launch, Fable 5 accounts for about 11% of total enterprise spending on Anthropic models, breaking the previous trend of customers quickly moving to the most powerful model.

According to analysts and investors, the development is mainly due to Fable 5’s high price and the fact that cheaper models are good enough for most tasks.

For example, Anthropic’s smaller, cheaper Opus 5 model has surpassed Fable 5 in enterprise spending since its launch in late July. Anthropic itself has declined to comment on the data.

It took nine seconds for an AI agent to destroy PocketOS’s production database. At work on a routine task in April, the coding agent, a variant of Cursor running on Claude Opus 4.6, ran into a credential mismatch and decided to fix the problem by triggering an API token. Little did PocketOS founder Jer Crane know that its activation would also delete its production database. “Had we known,” Crane later wrote on X, “we would never have stored it.”

The consequences of the agent’s actions were immediately apparent. Not only were recent backups belonging to PocketOS’ infrastructure provider contained in the production database — the recoverable versions were at least three months old — but so were those belonging to its infrastructure provider, Railway, which at press time still couldn’t tell Crane whether full infrastructure-level recovery was possible. Crane couldn’t fathom why the agent did this. So he asked it.

What he got back was an apology, of sorts. “I guessed that deleting a staging volume via the API would be scoped to staging only,” the agent said. “I didn’t verify. I didn’t check if the volume ID was shared across environments. I didn’t read Railway’s documentation on how volumes work across environments before running a destructive command.”

Ignoring built-in safety guardrails is hardly unique to agents operating on Claude Opus 4.6. In July, a Brazilian software engineer claimed an agent powered by OpenAI’s GPT-5.6 Sol model also deleted his production database, while in February, a Meta AI security and safety researcher claimed she had to switch off her computer to prevent an experimental agent deleting her entire inbox.

It wasn’t meant to be like this. Agentic AI was intended to be the culmination of millions of hours of research and development in gen AI to perform hyper-qualified acts of pattern recognition in the real world, and truly live up to their labor-saving promise. Their apparent predilection for destruction, however, has revived multiple debates about exactly how they should be restrained, and who, ultimately, is responsible for doing so.

Ultimately, the answer is those who green-lit the offending system. But as the pace of AI development puts greater daylight between companies pressured to adopt it, and those very tools capable of wreaking havoc across their internal databases, are CIOs now out of their depth?

Setting the pace

There’s no question the emergence of gen AI has changed the CIO role. “A few years ago, most of my time went to infrastructure decisions, including what to build, what to buy, and how to sequence the roadmap,” says Mike Trkay, CIO at data analytics company FICO. “Now, a growing share goes to questions of trust, verifying that when AI writes code, makes recommendations, or acts on behalf of a system, those actions can be explained and traced back to someone accountable for them.”

So the CIO has become the enterprise’s technological organizer du jour. “AI is accelerating software development, decision automation, and organizational experimentation at a pace that can outstrip institutional coherence,” says Edosa Odaro, executive advisor for data and AI at consulting firm VDS Global. “As AI becomes embedded across every business function, CIOs are increasingly responsible for ensuring that technical capability, governance, data quality, cybersecurity, human capability, and business strategy continue to evolve together rather than fragment.”

Day to day, that’s led to an exponential change of pace. “Things have always been fast,” says Zach Lewis, CIO and CISO of the University of Health Sciences and Pharmacy in St. Louis. “But now that speed of change is quicker, and you have to adapt.” And the need to catch up is constant. There’s no other option because then any competitor or co-collaborator can jump ahead, adds Lewis.

The rapid pace of change in AI also threatens to diminish the authority of individual CIOs who fail to keep up or set effective guardrails on those individuals who like to experiment with the newest models with loose regard for corporate security. “There’s all these AI tools that employees can now just go out and adopt,” says Lewis. And at the moment, a paid subscription to Claude or ChatGPT isn’t required to capitalize on its abilities. Consequently, staff are just a click away from asking LLMs to perform various tasks and expose sensitive corporate information in the process. “Everyone wants to play with the new thing,” he says. “And when they find benefit there, they’re going to want to bring it to their work lives.”

Agentic AI poses an entirely new set of problems. For one thing, says Odaro, the next phase of application adoption will be defined less by the capabilities of individual models, and more on what you allow their agents to do. “As AI becomes increasingly capable of generating software, coordinating workflows, and making recommendations across functions,” he says, “the challenge shifts from building AI to continuously governing evolving AI systems.”

This, Odaro continues, means that the CIO’s current approach to governance isn’t sustainable. “Static policies, annual reviews, and isolated oversight will struggle to keep pace with dynamic AI environments,” he says. “CIOs will increasingly need continuous governance capabilities that provide ongoing visibility into AI performance, value creation, risk, trust, and organizational adoption.”

Falling over the guardrails

How, then, should CIOs approach writing these new guardrails? Traditionally, this would be perfect fodder for so-called alignment researchers investigating how to instil a sense of morality and propriety into agents. According to analysts at Google DeepMind, however, it’s best to assume the agent will always be a potentially chaotic force within the company, and set parameters on its conduct from there.

“We borrow a lot from security, which already deals with the threat of internal employees who might be malicious, and we can apply these to a new setting,” Rohin Shah, Google DeepMind’s AGI safety and alignment team lead, told Fortunein June. Even so, he added, “AI is systematically different from humans.”

That difference primarily pertains to authority and speed. For agentic AI to live up to its full potential, it requires the freedom to access multiple systems simultaneously — an uncomfortable fact for CIOs hoping to align agent responsibility across the enterprise. In a time when workflows are becoming ever-more automated, however, that aspiration may prove unrealistic. In that case, Google DeepMind theorises that yet another monitoring layer for agentic AI may be required to make sure these free-roaming agents don’t cause too much trouble.

If that sounds daunting, you’re not alone. According to recent research by Gartner, up to 40% of enterprises using agentic AI will either demote or decommission these applications because their guardrails have proven inadequate. Preventing this, the research organization advises companies will need to adopt a graded approach to access, with autonomy for AI agents governed by the level of authority actually determined by the task they’ve been assigned.

Trkay is doing something similar at FICO. “Rather than chase every new model or capability, I focus control on the decisioning layer beneath it,” he says. “That includes the rules for what data AI can access, what it can act on autonomously, and where a human must sign off.”

All this, he adds, is defined from the start by a cross-functional governance committee, clear RACI ownership across standards and monitoring for the application, and a platform approach that enforces responsible AI usage. “Built well, that layer doesn’t need to be rebuilt every time the technology shifts,” says Trkay. “New capabilities plug into an existing structure of accountability, which is the difference between reacting to AI and running it.”

For his part, Trkay is skeptical that rigid guardrails can effectively restrain agentic AI from its most destructive impulses. “They tend to get worked around, either because they slow teams down or they’re too inflexible for legitimate edge cases,” he says. Effective guardrails for agentic AI, he adds, have to be specific enough to be meaningful, and adaptable enough to hold up as use cases multiply, backed by strong architecture, testing, and ongoing monitoring. “The one non-negotiable is the audit trail,” he says. “Whatever autonomy a system has, we need a record of what it did, and why.”

Staying grounded

For CIOs who don’t relish the challenge of setting obstacles and passing points for AI agents scurrying through their maze of networks, there’s always the option of delaying the inevitable by not immediately deploying such applications. Some might not even have the choice, at least for now. “We’re seeing the cost of tokens go up with those new models, because they’re expensive to run,” says Lewis. “But as new models come out, we’re going to see that decrease for some of those older models that were good.”

There is time, then, for CIOs to learn how to keep their head above the torrent of ever more new and powerful agentic AI applications. Whether they’ll be capable of doing so when the next great innovation is sold by Silicon Valley is an open question. Colin Constable, CTO of software development firm Atsign, styles himself as an internet optimist. Even he, however, is dismayed by the decreasing number of junior developers succeeding their more senior counterparts as they retire. That’s a big problem when so many of the former are relying on AI to assist them at work.

“We hand over lots of these decisions to LLMs without making good architectural choices,” says Constable. “If you haven’t been burnt by these things in the past, how would you know the difference?”

For their part, Constable and his colleagues get around this problem with a combination of AI-on-AI oversight of code quality, maintenance of constant dialogue within the team about new coding quandaries, and letting senior developers teach junior counterparts about some of the more avoidable mistakes in their profession. It’s a way of adapting to AI acceleration that points, unequivocally, toward CIOs diffusing responsibility for deeply educating the business about the technology. And if they continue to get it wrong, at least the agent will apologize.

When generative AI (GenAI) first arrived, I was in charge of a large team of data and machine learning engineers. We had built a full ML platform from scratch and had dozens of models in production delivering measurable results. AI was working for us.

But with the novelty of GenAI came the hype. Under pressure from the board, the question was no longer whether the technology could help our product, but how fast we could find a place for it. Our recommender system, which sat at the core of our company’s product, was the obvious candidate.

I pushed back. The Large Language Model (LLM) was trained to predict text, I argued, while our recommender was trained to predict user engagement. The LLM had never seen our proprietary interaction data and had no training signal on our objective. It could not know what our users clicked on, saved or abandoned, because it had never seen it.

It was a close call, but the pushback worked, and our attention moved to other problem spaces where GenAI was a legitimately strong solution.

It doesn’t always work out that way. There needs to be someone in the room who can translate the business strategy into the right technical decision.

I recently talked with the engineering team at a company I used to work for. They were building agentic AI infrastructure, but their algorithms weren’t winning. Under the same hype and the same leadership pressure, they had replaced ML models we built years earlier with AI agents. The results were unreliable, latency was much higher and outputs lacked a confidence score. Worst of all, the token usage was prohibitive.

Even the most powerful tool fails when applied to the wrong problem. LLMs are arguably the most powerful technology ever built, but that does not make them the right tool for every job. Choosing where a problem sits along the continuum from deterministic logic to machine learning to LLMs to fully agentic systems is the engineering skill that separates demos from production.

LLMs for explanation, ML for calibration

I’ve made that mistake too. Our team was building a content moderation model that would flag when customers posted content that was inappropriate or violated our terms of service. After a quick prototype, we decided that this was a perfect application of GenAI, because unlike ML algorithms, which only assign a probability, LLMs could explain why the message was flagged.

As we moved the model closer to production, we hit a snag. The probability value provided by the model was not just useful; it also needed to be accurate. That number was how we decided which content should go to manual review. But we found that self-reported probabilities from LLMs were uncalibrated and essentially unusable in practice.

Eventually we built a solution that leveraged both: the ML model provided the probabilities used for routing, and the LLM supplied the explanation that humans were able to interpret. The answer was to split one job into two, and use the right technology for each.

Simple logic often beats the most powerful agentic system

A key principle when designing agentic systems is to use LLMs as a last resort, and apply deterministic logic everywhere you can. The goal is to reduce the likelihood of an unnecessary mistake, thus increasing overall reliability.

Nowhere is this principle more overlooked than when connecting AI agents directly to the database. Querying a database to pull metrics is often done via SQL, and LLMs can do that by generating SQL on the fly. The problem is that SQL generation is probabilistic, and the results can change from one run to the next. You end up with an agent that confidently returns the wrong number, which introduces the need for human review and defeats the purpose of automation.

There’s plenty of data to support this claim. A 2024 study by Ouyang et al. ran 829 coding problems through the same model five times and found that up to three quarters of them produced no semantically identical results. For data warehouses specifically, the BEAVER benchmark, built by researchers at MIT, Harvard and other renowned institutions, shows that off-the-shelf LLMs perform poorly when querying enterprise data, partly because that data is private and models have never trained on it and partly because of the complexity of real enterprise environments. As of early 2026, the top execution accuracy on the leaderboard is 11.4 percent.

This is why the semantic layer is one of the most useful assets when building agents in the real world. It replaces query generation with a simple retrieval, where the SQL is built by the backend engine in a deterministic manner.

Whether using a semantic layer or building MCP tools that hard-code query parameters, deterministic logic will almost always beat probabilistic SQL generation when querying a database via an agent. Business logic should be written and validated once, not regenerated probabilistically each time.

LLMs solve the cold-start problem

Search systems are complex pieces of engineering. Sophisticated real-world implementations involve several steps, from classifying intent to candidate retrieval, to ranking. Here is where LLMs provided a clever solution that saved us a ton of time when combined with an ML algorithm.

Let’s take intent classification. Imagine multiple categories of products that can be retrieved from the same search bar. Classifying intent here means determining which category the user is searching for, which can be genuinely ambiguous. Our solution was to build one classifier per category, which requires labeled data we didn’t have.

Collecting the data was out of the question for us. It would have been too expensive, so we turned to LLMs instead. It worked great. The classification was high quality, and we felt that we no longer needed labeled data to train an ML model.

But there was a catch. Latency was prohibitive, and so was the token cost. So, we landed on a hybrid approach. We used the LLM to generate labeled data to train an ML classifier. We got the best of both worlds, solving the cold-start problem while avoiding the latency and costs of large language models. That’s the system that ultimately won in production. The LLM earned its place at build time, not at request time.

The continuum, and how to choose

It is tempting to take a powerful solution, such as LLMs and AI agents, and simply apply it to every problem. But when we look at the four use cases above, each of them led to a different decision. In the recommender system, GenAI did not earn its place. In content moderation, it earned part of the job, providing the explanation while the ML model provided the score. In querying the database, the agent did the reasoning, while the semantic layer pulled the right number. And in search, the LLM solved the cold-start problem, but it was never deployed to production.

Behind all of these decisions is the same underlying question: where does this technology earn its place, and does it justify the added complexity?

Production systems often combine multiple levels of algorithmic intelligence. A customer service agent uses an ML model for query routing, a semantic layer for metric retrieval, an LLM for drafting a reply and deterministic logic as enforced guardrails. Four technologies on the continuum, each earning the complexity it brings.

Business pressure is a powerful force, and it can cloud engineering judgment. As leaders, we must filter through the hype and look for the problems where a new technology genuinely adds value, rather than treating it as a tool that will solve all of them. The careless decision satisfies the board but hurts the company; the thoughtful one creates lasting value and still earns the board’s approval.

With vision comes perspective, and over a distinguished career, IT and digital transformation leader Niraj Bhatt has held may titles, and earned three consecutive CIO 100 awards since 2023.

As a storied advisor for startups and Fortune 500 companies, helping them navigate the unpredictability and fluidity of AI, Bhatt knows how emerging tech is rapidly reshaping the way organizations build products and deliver value, and how challenges shift as companies move from experimentation to real-world deployment.

AI, of course means a lot of different things to different people, and also for frictionless startups and large enterprises. For the former, speed is a huge asset, allowing them to punch above their weight. But it also means they need lightning fast reactions when landscapes shift. “The same speed can also hurt them when larger AI companies release new offerings that disrupt what startups are building,” he says, referencing recent moves by Anthropic and Google.

On the enterprise side, the conversation is more about scale and risk. Many large organizations have moved past the POC stage and now wrestle with the realities of putting AI into production.

Cost for both is naturally a recurring theme as organizations scale up AI efforts, and true expenses become clear only after the initial excitement fades. “Every input and output token, and the model you’re selecting, add up,” he says. Some customers like Open AI, he adds, get throttled because their usage, volumes, and costs are growing so fast, making planning, observability, and monitoring critical for any team moving beyond experimentation.

So understanding the full software development lifecycle is also vital. Therefore, before committing to production, he helps clients see the big picture, and make sure they understand technical requirements as well as operational and financial implications. “The cost picture isn’t just about usage, but scale and the model choices teams make,” he says.

Bhatt also discusses effective approaches to AI and enterprise IT, technology leadership, and the evolving role of today’s CIOs. Watch the full video below for more insights, and be sure to subscribe to the monthly Center Stage newsletter by clicking here.

On AI hype: If you can’t explain something to someone who’s eight or 80, you don’t really understand it. It’s gone from LLMs, to RAG, to agentic AI, and now the essence is all about tokens. It’s predicting that next token and understanding that is key. So when LLMs came out, they were good at doing that on the data on which they were trained. When the enterprises looked at it, they wanted to make those LLMs work for their data. And the question became how to provide our data and context. It’s about building the right context for the LLM. Agentic AI is similar and that’s where the RAG evolution came in, in that I’ve got my data because every LLM has limitations in terms of how much context it can carry.

There are ranges of LLMs, where Google has the highest in regard to the context window size and what they support. Agentic AI is more action oriented, though. LLMs rely on the metadata you provide for the tools. Then they’re doing token prediction in that whatever I’m looking for, I should use a specific tool. Then it’s the infrastructure underlying which LLM it relies on to invoke the agent. So if you try to explain the microservices to a person, you’re going to struggle. But it’s very important to understand the evolution and that’s where you can cut through the hype. Understanding in this context is key.

On navigating challenges around talent: What I’m seeing on the IT side is there’s so much cognitive load, so how do we empower people to build solutions with the right mix of products and platforms? I think it’s about democratizing AI for the entire organization. Your talent strategy is everyone, all inclusive, starting from interns, the business and tech sides, CEO, everybody.Like your customer success or revenue officers, you need a talent strategy because in the end, IT alone isn’t going to be in a position to deliver for everyone in the organization.

AI has the potential to make everyone in the organization more productive. You have to plan that and facilitate broad innovation across the organization.That’s where the talent strategy, and working with HR and the people officer becomes very important providing those tools. One part of it is training, but how do I build an agent for a receptionist receiving calls, for instance?I’m not going to rely on vibe coding or things of that nature. But what are the tools? Where do I go, where do I host this? I think through that entire ecosystem beyond copilots. That’s where innovation can kick in, and that broader talent strategy is something I’m working with my customers on.

On collaboration: I heard a panel discussion recently, and a question was asked about what’s the number-one trait CIO needs to be successful at in the world of AI, and the answer was collaboration. You need to bring everybody together, move forward together, and make sure everybody’s on board. And in my mind, simplifying that is more like systems thinking when you operate, just bringing everybody along and ensuring they’re meeting outcomes.

But maybe what’s more important is managing expectations. Because if you’re a CIO, there’s a tremendous amount of pressure to deliver and have a rock solid AI strategy. So what I’m doing with my customers is get the board, CEO, and CFO into a room and help them understand what I’m talking about, the evolution, and what’s the art of possible. You don’t want to be a CIO who thinks I have a hammer and everything is a nail. Having buy in from the senior leaders is essential to know you’re headed in the right direction. You’re not reacting to pressure from top leadership, but driving and becoming the change agent for good for the company.

On navigating AI: It’s interesting times. I’m covering a spectrum of startups, non-technical and technical founders, and advising Fortune 500 companies. What I’m seeing is they love the velocity and momentum because that’s what they’ve always wanted, and AI is providing that. They’re able to bring their products to markets very quickly, so something that would’ve taken three years a couple of years ago is probably now taking them three months. There’s a lot of excitement there. But on the flip side, the same velocity is also hurting them. There are so many frontier AI companies getting disrupted. OpenAI, for instance, has offerings in sales and marketing, and Google has an interactive video model. So a lot of startups working in the marketing space are getting stuck. A lot of what I’m focused on is working with founders, helping them pivot in the gen AI space, ensuring their systems and products are built and structured in the right manner.

And on the enterprise space, what I’m seeing is the POC wave, and people have seen the value. There’s some excitement but now the struggle is getting them to production. That’s where you run into cost, latency, legal compliance, privacy issues, and customer concerns that if we get tickets to production, how’s it going to look and how are we going to scale. So engineering and product teams have to be ably supported by the enterprise architecture and R&D teams. I then help them get up to speed and build that internal platform product for the production workloads. It’s exciting times on both sides.