Businesses received a staggering amount of cyberattacks in June, according to Check Point, showing a rise of 20% over the previous 12 months. The breakout of AI agents from OpenAI in July to hack into the Hugging Face website, and subsequent similar events from Anthropic and Meta, indicate agentic-powered attacks will explode over the coming year.

Currently, malicious hackers have the advantage because publicly released frontier models from the US incorporate guardrails that can’t distinguish between malicious or defensive activities. As a consequence, these models default to a refusal to get involved. Hugging Face discovered this the hard way when they attempted to utilize a model to defend against the OpenAI intrusion. Their solution was to adapt a Chinese open weight model to analyze the 17,000 attack logs, find the vulnerability, and contain the intrusion.

With incidents like these happening more often, an arms race has begun with AI being both the problem and the solution.

Strength in numbers

While single agents generally perform more efficiently for well-defined tasks, research from Stanford University indicates swarms are more effective in messy scenarios with noisy data, which are more typical of unpredictable, intrusion attacks. The increased token usage by swarms raises costs, but increasingly efficient open weight models are rapidly lowering these barriers.

In the Hugging Face example, the agents worked together as a team leaving messages for each other on a message board they improvised. They shared newly found vulnerabilities, exchanged tools, and even developed conventions to address one another and to avoid overwriting each other’s work. While this may seem sinister, they were only following their designated purpose: to achieve a goal without regard to any collateral damage. We can expect bad actors to harness the power of agentic swarms through fine-tuning open weight models, and creating agents that progressively learn from their experiences.

Modern warfare has been transformed over the last four years, too, through the deployment of drones by Ukraine to defend against Russian attacks. Military strategies and the deployment of armament budgets around the world are shifting to focus on new technologies, and approaches and enterprises are now facing a similar challenge from the hostile use of agentic AI.

The drawbridge is down

As enterprises build out their own agentic systems to handle ecommerce, customer service, and marketing activities, this presents new attack surfaces for antagonistic efforts. April 2026 research from Trend Micro found almost 1,500 MCP servers directly exposed to the internet had no authentication or encryption, a rise of 200% from nine months earlier. This included 70 hosts offering direct SQL execution, and servers holding medical records.

The automation of business processes and the reduction of humans from decision making chains open up new vulnerabilities for agents with malicious intent. Arkose Labs’ 2026 agentic AI survey of 300 enterprise leaders found 97% expected an AI agent security incident within the next 12 months.

Social engineering

While agents have demonstrated their ability to break through security systems, they’re also capable of targeting humans to achieve their objectives. Recent research from Verizon indicates that 62% of successful breaches involve a human element, with phone-based attacks 40% more successful than email-based ones. In August, for instance, scammers using an AI-generated deep fake of Australian Prime Minister Anthony Albanese’s voice were able to scam investors out of $5.3 million.

If agents can break out of digital sandboxes, and generate convincing fake videos and audio, then they’re certainly capable of making basic phone calls. In July, during testing of frontier models, the UK AI Security Institute discovered an agent tried to insert malicious code into an open-source project. Attempting to get the code approved, the agent created fake online identities using them to persuade the project’s maintainer to sign it off. “This is the first time we’ve seen risks around autonomy and deception manifest this clearly without specific prompting in the real-world,” the Institute put in a write-up of the incident.

Fight AI with AI

So attackers currently have the upper hand in this escalating arms race. They have access to agents that can work around the clock, constantly probing, learning, and sharing their knowledge with other agents. They’ll only get better at this and learn ways to stay ahead of defensive systems. International agreements to delay or restrict the capabilities of frontier models won’t stop hostile actors motivated by money or rogue states pursuing other objectives. Developers and security vendors need access to the latest frontier models unfettered by restrictive guardrails if we’re to stand any chance of defending against the coming tsunami of attacks.

We can learn a lesson from recent history on this front. In 1992, the US restricted exported software to weak 40-bit encryption, citing security concerns going back to the cold war. While the US allowed stronger encryption internally, the result was weakened security for everyone as hostile antagonists were able to disrupt global supply chains that incorporated less secure software. Despite lifting the ban in 1999, embedded software containing 40-bit encryption continued to cause problems for many years across multiple countries, including the US.

Without rapid action, we may look back fondly to the world before July 2026 as a golden age for cybersecurity, a relative age of innocence.

The CIO job description is being rewritten in real time. As AI agents take over the interface layer and connect directly to any data source, the skills that once defined great IT leadership — UX fluency, applications integration, build-versus-buy judgment — are giving way to an entirely different set of questions surrounding not how a process works, but whether it needs to exist at all.

Thimaya Subaiya is living that shift firsthand. At Cisco, he oversees IT and says the ideal CIO candidate today might not have a traditional IT background. Here, he explains why he split the company’s AI leadership out as its own function and why he’ll merge back in, what he’s really looking for in a CIO candidate, and why the Cisco CIO job is such a good one.

How would you describe your role at Cisco?

I lead operations for one of the world’s largest supply chains, as well as security and trust, including product security, internal systems, and data center security. I also lead the CIO organization and have revenue operations, partnership management, and accountability for our AI strategy. Two and a half years ago, I consolidated AI from throughout the company and named a CAIO. I then split out the role to give us a boost in the AI space, but eventually, the CAIO role will merge into IT.

How did you conceptualize the CAIO role?

At first, it was a leader who could pull use cases from all our operations and execute. The role also included the ethical use of AI systems, and prioritized what to guardrail and push out to employees.

But it’s evolved. To take a step back, Cisco pioneered enterprise networking, then built Compute with Cisco, Storage with Cisco, Networking with Cisco, Security with Cisco, and Observability with Cisco. Today, the CAIO is moving up the stack with an AI framework for MCP connectors, which has really moved us forward.

This CAIO group can tell the Cisco-on-Cisco story for AI, because we have a testbed for new ideas. If we continue to rely on multiple vendors, as in the past, we won’t be able to integrate at scale. This is why we isolated the CAIO role, to focus exclusively on AI governance and execution.

You’re in the middle of a CIO search. What are you observing about the CIO talent market?

With AI, the CIO role has completely changed. It’s no longer about UX and applications integration because with MCP, we can connect to any data source at any time, and agents have replaced the interface. The CIO role is now more about rethinking a process and then deploying an agent to execute, rather than reworking a process.

So the ideal CIO is a traditional one who’s learned to think differently, or even someone without a CIO background, but who’s led in product management, innovation, or transformation. The role today requires someone who’s been disruptive, and has had to rethink how a company operates, not just how its applications work.

Our top criteria are strategy, speed of execution, and the ability to scale because we’re not investing in science projects. For example, when the sales team requests a better forecasting tool, a CIO traditionally would make a build or buy decision. But in today’s world, the right question should be if you need a solution to forecast at all, or can an agent do it. Or better yet, do we even need this process?

So what’s the right background for today’s CIO?

Product managers have a relevant background because they manage multiple aspects of how a product comes together: user needs, business outcomes, fit in the market, and getting it built. This understanding of product strategy, marketing, and adoption is extremely important right now because we treat our AI initiatives like products. So a great path for our CIO is data scientist foundations, product management, and transformation.

What about enterprise security?

I treat enterprise security as a separate organization, which every company should do. Testing and evaluating new cyber solutions for frontier models requires a lot of work like scanning everything, taking a neutral view of what’s broken, deciding which tools become standard within development frameworks, which cryptography tools to use, and then maintenance. Abstracting that into its own organization creates focus. It also lets us move at the speed of AI.

When AI attacks, you need AI to defend you, and if security is embedded within the CIO organization, it’s not top of mind for the business. Security has become its own board-level conversation. For today’s CIO, I’d keep AI in but take security out.

A year after the CIO is in place, what will success look like?

Our applications footprint has been reduced, we’ve seen pure productivity gains from accelerating the back, and the speed of new releases is increased. The team is becoming more effective with the same resources, and we can say that our CIO drove us to leverage everything new technologies offer without blowing up on tokens. We’re looking for a new way to operate IT.

Why is the CIO job at Cisco a great opportunity for the CIO you’re describing?

It’s possibly the coolest job out there. We have an entire AI stack end-to-end that nobody else can claim because we bring networking and security together, complemented by observability and collaboration. That combination means we can create net-new solutions that define what technology looks like in the future.

On the security side, we’re one of the very few companies truly integrating AI into defense in a way that can be leveraged across a much broader market. That’s exciting, because it means free access to an entire stack that lets you innovate in ways the industry hasn’t seen before.

I call AI today’s generational technology. Every generation gets a technology that redefines how it operates, including the internet, iPhone, and now AI. Cisco is about to become the first company to launch a personalized AI agent for every employee, reachable through Webex. Think of it this way: the average person has an IQ of around 100. Now every employee is paired with an AI agent that can exponentially increase human capacity, built entirely on the technology available today.

Getting to build things like that, with no proven methodologies or limitations, and nothing but the question of how we get to the future, is the most exciting thing there is if you’re an innovative leader.

For the past few years, enterprise AI programs have focused on making models more useful, accurate, and autonomous. In that phase, a bad answer was still usually something a human could accept or reject before taking action. But once agents start invoking tools and acting inside business workflows, success should no longer be measured only by how much work they complete. A more important metric is how well an agent recognizes when it lacks the authority, context, or judgment to continue.

When helpful becomes risky

According to Allan Dabre, technology compliance and AI lead at PwC, a behavior that has to be deliberately designed into the system is, “I don’t know.” AI is built to be helpful, so an agent will generally try to do something useful unless it’s been configured not to.

“The fact that AI systems can hallucinate illustrates that tendency,” Dabre says. “When they lack enough information, they may still produce an answer. In an agentic workflow, that impulse can become more dangerous because the output may become an action, rather than remain a suggestion.”

He adds that many enterprises still test AI primarily for completeness and accuracy. That made sense when the central question was if the model could produce a reliable response. But as models improve and agents gain more operational authority, he argues that CIOs need to prioritize something else: restraint.

“Can it stop at the exact moment you want it to stop?” he asks. “Are you testing for that?”

Confidence is not authority

Dabre makes a simple but important distinction. An AI agent may be 99% confident a record should be updated, a refund should be approved, or a legacy database can be decommissioned. But that doesn’t mean the agent has the authority to act. Confidence is about the probability the system believes it’s right. Authority is about whether the organization has delegated that action to the system in the first place.

He gives the example of an agent asked to analyze legacy software and recommend what can be decommissioned. The agent may conclude, with high confidence, that several databases have little user impact and can be deleted. But even if the system is confident, most organizations wouldn’t want it to delete those databases on its own.

The same logic applies across business processes. An agent may be confident a customer record should be updated, an opportunity in a CRM system should be closed, or a transaction appears legitimate. But once that action flows into other systems, the potential consequences expand.

That’s why Dabre argues for what he calls an agent harness: a controls or orchestration layer outside the model that defines what the agent can and can’t do. In a refund workflow, for example, a company might let the agent approve small refunds, require human approval for larger ones, and stop the process entirely above a defined threshold. The agent may gather the relevant context, explain the request, and prepare the case for review, but the decision is governed by the authority boundary encoded into the system.

“It’s not a policy document and it’s not a prompt,” Dabre says. “It’s software or a configuration you can apply to an agent.”

The case for least agency

Matt Graney, chief product officer at Celigo, a business automation and integration platform provider, approaches the same problem through a principle he calls least agency. The idea is to give an agent the least amount of autonomy required to complete a job.

According to him, there’s a temptation to throw AI at broad, nebulous problems. But many business processes are still largely deterministic. They follow established rules and perform repeatable work. Within those workflows, AI may be useful at the point where rigid rules give way to interpretation. But that doesn’t mean the agent should own the entire workflow. “The smaller you make that surface area, the better,” he says.

Graney says the same logic applies to tools. An agent with too many tools can become confused, especially as context windows grow and the task becomes more complex. “Because Celigo is an integration platform,” Graney says, “the company’s approach is to expose agents to fewer, more powerful tools that reach enterprise systems through governed connections.”

That’s another form of restraint. Instead of letting an agent reach into enterprise systems ad hoc, the business gives it a narrow, governed toolset designed for the task at hand.

Graney also argues that guardrails should sit outside the model. If the same agent that makes a decision is also responsible for judging whether the decision is acceptable, the control is weaker. A separate guardrail can check the agent’s inputs and outputs before a downstream action occurs.

That same design discipline applies to escalation. “I don’t know” shouldn’t be treated as a chatbot phrase. In an enterprise workflow, it’s a handoff path that should be defined before the agent reaches it.

Make escalation part of the workflow

Turning uncertainty into a handoff is where Matt Quinn, CTO at CarGurus, an automotive marketplace, sees agentic AI becoming less a pure technology challenge and more a management challenge. At CarGurus, Quinn says agents are evaluated according to what they know, what they can do, and what data they operate on.

CarGurus receives a high volume of cases from dealers, and each one needs to be classified and routed. The company now uses an agent to review incoming cases, draw on account history, and route them to the appropriate next step. Quinn says the agent handles about 70% of those cases end to end without human involvement.

But when agents move toward consequential actions, he says the consensus is having a human approval step. The agent may return with a simple prompt like, I’m about to do this. Do you want me to proceed? That simplicity matters because a handoff shouldn’t bury the reviewer in complexity.

Quinn says the human remains ultimately accountable for the work. That principle is especially important in engineering, where agents may help write code or fix bugs. Quinn adds that CarGurus still expects engineers to follow the practices they’d use for any other production change, which includes running quality checks.

The company has adopted the phrase healthy speed to describe the balance it wants. The goal is to move faster without letting quality degrade. An agent can accelerate work, but if teams abandon the practices that make work safe, the speed becomes reckless.

This is also where human judgment remains difficult to replace. Quinn describes it as high judgment people develop through experience. A human may look at an AI-generated output and sense something’s wrong, even before fully articulating why. “Agents are improving,” he says. “But humans still play a critical role in deciding when the system shouldn’t continue.”

That doesn’t mean every workflow needs the same level of review. Quinn says CarGurus doesn’t have a target percentage of work to automate. The right level depends on the job and the task. A simple bug fix may require a lighter review than a change to a sensitive backend service, and a personal summary may carry little risk. But a document sent under someone’s name still needs human review.

Make autonomy accountable

That kind of pragmatic approach may be the best lesson for CIOs, making the goal of agentic AI appropriate rather than maximum autonomy.

That also means ownership has to be clear. Dabre argues ownership should be divided before deployment. The business defines the outcome, technology builds and configures the agent, risk and compliance set the guardrails, and governance monitors whether the system still behaves as intended. The authority to pause, stop, or retire an agent should be defined before production, not negotiated during an incident.

Graney makes the same point with a simple analogy. If a company hires an untrained intern, gives that intern access to the crown jewels of a business process, and something goes wrong, the intern isn’t the real problem. The process is. The same applies to agents. Accountability belongs with the person who owns the workflow.

That may be the shift CIOs need to make as enterprises move from pilots to production. AI agents shouldn’t be treated as magical workers that absorb accountability. They’re components in business processes, and those processes need accountable owners.

As AI adoption increases, the next phase of enterprise maturity won’t be defined by agents that always answer or always complete the task. It’ll be agents that know when not to act.