Visualização normal

Ontem — 8 de Maio de 2026Stream principal
  • ✇Security | CIO
  • “채용이 곧 공격 경로”…AI 악용한 가짜 IT 인력, 기업 내부 위협으로 확산
    최근 몇 년 사이 가짜 IT 인력을 채용하는 문제는 점점 심각해지고 있지만, 이를 공개적으로 인정하려는 기업은 많지 않다. 포춘 500 기업부터 중소 조직에 이르기까지 원격 채용 방식이 악용되면서, 실제 신원이 아닌 인물에게 신뢰 기반 접근 권한이 부여되는 사례가 발생하고 있으며 이는 내부자 위협으로 이어질 수 있다. 추정에 따르면 미국 전역에서 수천 명의 가짜 IT 인력이 활동 중이며, 이들은 정보와 지식재산(IP), 데이터 탈취는 물론 해외로의 업무 외주화, 시스템 교란, 외국 정부로의 자금 유입 등 다양한 위협 행위를 수행할 수 있는 위치에 있다. 미국 기업 아마존(Amazon)의 최고보안책임자(CSO) 스티브 슈미트는 “북한이 IT 직무를 확보하기 위해 시도한 1,800건 이상의 사례를 차단했으며, 그 수는 계속 증가하고 있다”고 밝혔다. 일부는 개인적인 이익을 위해 미국 직원으로 위장하고, 또 다른 경우에는 북한과 같
     

“채용이 곧 공격 경로”…AI 악용한 가짜 IT 인력, 기업 내부 위협으로 확산

8 de Maio de 2026, 03:58

최근 몇 년 사이 가짜 IT 인력을 채용하는 문제는 점점 심각해지고 있지만, 이를 공개적으로 인정하려는 기업은 많지 않다. 포춘 500 기업부터 중소 조직에 이르기까지 원격 채용 방식이 악용되면서, 실제 신원이 아닌 인물에게 신뢰 기반 접근 권한이 부여되는 사례가 발생하고 있으며 이는 내부자 위협으로 이어질 수 있다.

추정에 따르면 미국 전역에서 수천 명의 가짜 IT 인력이 활동 중이며, 이들은 정보와 지식재산(IP), 데이터 탈취는 물론 해외로의 업무 외주화, 시스템 교란, 외국 정부로의 자금 유입 등 다양한 위협 행위를 수행할 수 있는 위치에 있다.

미국 기업 아마존(Amazon)의 최고보안책임자(CSO) 스티브 슈미트는 “북한이 IT 직무를 확보하기 위해 시도한 1,800건 이상의 사례를 차단했으며, 그 수는 계속 증가하고 있다”고 밝혔다.

일부는 개인적인 이익을 위해 미국 직원으로 위장하고, 또 다른 경우에는 북한과 같은 국가 단위 조직이 자금 확보 및 기타 불법 목적을 위해 IT 인력으로 위장하기도 한다.

현재 AI 기술은 딥페이크 생성, 더욱 정교한 영상 면접 수행, 빠른 신원 변경 등을 가능하게 하며 이러한 위협을 한층 고도화하고 있다.

슈미트는 공격 방식 역시 변화하고 있다며, 단순히 프로필을 조작하는 수준을 넘어 실제 미국인의 신원을 구매해 활용하는 단계로 진화하고 있다고 경고했다.

사이버보안 기업 센티넬원(SentinelOne)의 위협 연구원 톰 헤겔은 “이 문제는 전통적인 의미의 채용 사기가 아니다”라며 “공격자가 ‘채용되는 것’을 첫 단계로 삼는 내부자 위험 문제”라고 설명했다.

CIO, CISO 등 IT 리더들은 가짜 및 사기성 IT 인력에 대해 지속적으로 경계해야 하지만, 조직이 이를 인지하지 못한 채 피해를 입는 경우도 적지 않다.

가짜 인력은 어떻게 채용을 통과하나

채용 과정에는 단일한 실패 지점이 존재하지 않는다. 가짜 및 사기성 IT 인력은 신원을 숨기고, 역량과 경력을 조작하며, 면접과 검증 절차를 별다른 의심 없이 통과한다.

센티넬원은 북한 연계 IT 인력 조직과 관련된 약 360개의 가짜 인물과 1,000건 이상의 채용 지원 사례를 추적했으며, 자사 채용에도 실제 지원 시도가 있었다고 밝혔다.

헤겔에 따르면 공격자들은 점점 더 대규모로 사회공학 기법과 신원 은폐 전략을 활용하고 있으며, 채용 과정은 이들이 침투하기 위한 핵심 진입 지점으로 작용하고 있다.

이들은 합성 또는 도용된 신원을 기반으로 이력서와 온라인 프로필을 만들고, 스크립트나 대리 응시자, AI 기반 응답을 활용해 면접을 통과한다. 또한 백그라운드 체크는 제출된 정보만 검증하기 때문에 이러한 조작을 그대로 통과시키는 구조다.

헤겔은 “가짜 구직자들은 이제 AI 도구를 활용해 실제 지원자를 모방하고 있다”라며 “초기 신원 검증을 통과할 수 있는 합성 신원을 만들고, 경력 이력을 조작하며, 실시간 AI 지원을 통해 면접에서도 설득력 있게 응답한다”고 설명했다.

보안 기업 플래시포인트(Flashpoint)의 조사에서는 HR 및 채용 플랫폼 계정 정보가 저장된 악성코드 감염 시스템, 번역된 면접 코칭 메모가 담긴 브라우저 기록, 해외에서 기업 장비를 원격 조작하는 ‘노트북 팜’, 그리고 가짜 경력 검증을 위한 페이퍼컴퍼니 등이 확인됐다.

문제는 채용 이후다. 채용이 완료되면 계정과 장비가 지급되고 시스템 접근 권한이 부여되면서 이들은 곧 내부 신뢰 인력으로 전환된다. 헤겔은 “장기적인 위험은 단순히 가짜 직원을 채용하는 데 그치지 않는다”라며 “기업 시스템과 민감한 데이터에 악의적인 접근을 스스로 열어주는 결과로 이어질 수 있다”고 경고했다.

가짜 IT 인력 대응 방법

CIO가 가짜 IT 인력을 의심하는 순간부터 문제의 성격은 단순 채용 이슈에서 내부자 리스크 관리로 전환된다. 이후 대응 절차가 무엇보다 중요해진다.

몽고DB(MongoDB) 재직 당시 조사 및 대응을 총괄했던 IANS 자문이자 베드록 데이터(Bedrock Data) CSO 조지 거초우는, 재직했던 회사가 북한 연계 가짜 IT 인력을 채용한 사실을 뒤늦게 인지하고 조사에 착수한 경험을 공유했다.

문제는 엔드포인트 보안 솔루션 제거 시도에서 시작됐다. 거초우는 “크라우드스트라이크 오버워치(CrowdStrike Overwatch)를 포함한 보안 기능을 제거하려는 시도가 감지됐고, 이후 해당 노트북이 북한 IP 주소와 통신하는 정황이 포착됐다”고 설명했다.

이어 “보안 도구 조작과 북한 연계 트래픽이 동시에 나타난 것은 일반적인 신규 입사자의 행동이 아니라는 명확한 신호였다”고 덧붙였다.

조사 결과 해당 인력은 도용된 신원에 AI로 생성된 이력서, 스크립트 기반 면접 답변을 결합해 검증 절차를 통과한 것으로 드러났다. 기존 백그라운드 체크는 제출된 정보만 확인할 뿐, 조작 여부를 탐지하지 못하는 한계가 있었다.

거초우는 “많은 검증 시스템이 조작된 경력, 합성 신원, 재활용된 개발자 프로필을 식별하지 못한다”라며 “이 때문에 별다른 경고 없이 채용과 면접을 통과할 수 있었다”고 설명했다.

이후 조사에서는 보안 도구 비활성화 시도, 장비 내 지속 접근 확보, 권한 상승 탐색 등의 행위가 확인됐다. 거초우는 “발각되지 않았다면 연방 보안 인증(FedRAMP) 환경까지 접근했을 가능성이 있다”며 위험성을 강조했다.

놓치기 쉬운 ‘경고 신호’…단편적 대응이 문제

사건 이후 돌아보니 여러 이상 징후가 있었다. 면접 영상 품질이 낮고 화면이 불명확했으며, 통화마다 억양이 일관되지 않았다. 면접 평가도 분산돼 있었고, 이를 통합 검토하는 체계가 없었다.

노트북 배송 주소를 마지막 순간 변경한 점도 주요 단서였다. 거초우는 “이는 ‘섀도우 워커’들이 자주 사용하는 전형적인 수법”이라고 말했다.

문제는 이러한 징후가 각각 개별적으로는 채용을 막을 정도로 치명적이지 않았다는 점이다. 거초우는 “각 이상 징후를 통합해 판단하는 역할이 없었기 때문에, 엔드포인트 경고가 발생하기 전까지 패턴을 인식하지 못했다”고 설명했다.

발견 이후 대응…즉각 차단과 전면 조사

가짜 인력이 확인되자 팀은 즉시 장비를 격리하고 모든 계정을 폐기했으며, 포렌식 조사를 실시하고 연방 당국에 신고했다. 조사 결과 데이터 유출이나 내부 확산은 발생하지 않은 것으로 확인됐다.

이후 대응 조치로는 채용 과정에서의 신원 검증 강화, 초기 이상 징후를 통합 관리하는 ‘옐로 플래그’ 담당자 지정, 신규 입사자에 대한 신뢰 확보 전까지 접근 권한 제한 등이 도입됐다.

“신원보다 행동”…채용 이후 모니터링 강화해야

거초우는 채용 이후 행동 기반 모니터링의 중요성도 강조했다. 단순 자격 증명보다 실제 사용 행태가 위장 인력을 식별하는 핵심이라는 설명이다.

이에 따라 기업은 보안 또는 HR 부서 내 검토 담당자를 지정해 면접 영상 품질 저하 등 채용 과정의 불일치를 식별해야 한다. 또한 AI로 생성된 링크드인 프로필, 이력서 불일치, 장비 배송 주소 변경 등도 주요 점검 대상이다.

패널 면접과 프로젝트 기반 평가를 통해 도용 또는 가짜 개발자 신원을 재활용하는 지원자를 식별하고, 신규 입사자에게는 초기 단계에서 민감 데이터나 운영 환경 접근을 제한하는 것이 필요하다.

또한 IAM, EDR, VPN 등 보안 에이전트가 비활성화될 경우 경고를 설정하고, 가짜 개발자 채용 상황을 가정한 탐지·대응 훈련도 병행해야 한다.

거초우는 “근무 시간 외 접근, 내부 시스템 전반에 대한 과도한 검색, 대량의 문서 및 코드 저장소 복제 시도 등도 주요 이상 징후로 주의 깊게 살펴야 한다”고 강조했다.

IT 리더들이 내부에서 목격하는 현실

고용 사기 문제는 앞으로 더욱 악화될 전망이다. 가트너는 2028년까지 전 세계 채용 지원자의 4명 중 1명이 가짜일 것으로 예측했다.

에너지솔루션(Energy Solutions)의 CIO 데이비드 웨이송은 “가짜 및 사기성 구직자의 증가는 조직 전반에 걸친 ‘전염병’ 수준으로 확산되고 있다”고 말했다.

웨이송에 따르면 공격자들은 데브옵스, 시스템 관리자, 데이터 엔지니어, 데이터베이스 관리자 등 높은 접근 권한을 가진 기술 직무를 집중적으로 노린다. 이러한 직무에 채용될 경우 핵심 시스템에 대한 깊은 가시성과 통제 권한을 확보할 수 있기 때문이다.

웨이송은 “이들 직무는 사실상 ‘성문 열쇠’를 쥔 역할”이라며 “시스템 접근을 노린다면 일반 개발자보다 훨씬 가치가 높은 목표”라고 설명했다.

규제가 엄격한 에너지 시장에서 운영되는 에너지솔루션은 미국 내 인력 채용과 데이터의 미국 내 보관이 계약상 의무화돼 있다.

웨이송은 가짜 IT 인력을 직접 식별한 경험을 바탕으로 다른 IT 리더들에게 경고를 전했다. 가장 초기 징후 중 하나는 비정상적인 지원자 급증이었다. 수 시간 만에 수백 건의 지원서가 몰렸으며, 이는 기업 인지도 대비 과도한 수준으로 자동화 또는 조직적인 활동을 시사했다.

면접 단계에서는 ‘신원 바꿔치기’ 사례도 확인됐다. 웨이송은 “전화 인터뷰를 통과한 사람과 화상 면접에 등장한 사람이 다르고, 이후 또 다른 인물이 나타나는 경우도 있었다. 모두 동일한 이름과 이력서를 사용했다”고 밝혔다.

문제의 근본 원인 중 하나는 기존 채용 절차가 정보와 역량을 개별적으로 검증한다는 점이다. 웨이송은 “전통적인 백그라운드 체크는 제출된 정보만 확인할 뿐, 사기를 식별하지 못한다”고 지적했다.

일부 CIO에게는 불편한 현실이지만, 이들이 수행하는 업무 결과 자체는 높은 수준일 수 있으며, 탐지는 성과가 아닌 이상 징후를 통해 이뤄지는 경우가 많다.

그러나 가짜 IT 인력은 보안 위험뿐 아니라 비즈니스와 규제 리스크도 동시에 초래한다. 특히 규제 산업에서는 계약 위반, 규제 조사, 고객 신뢰 상실로 이어질 수 있다.

웨이송은 “가짜 IT 인력은 보안 문제를 넘어 비즈니스와 컴플라이언스 측면에서도 심각한 위험을 초래하며, 규제 산업에서는 계약 위반과 규제 리스크, 고객 신뢰 훼손으로 이어질 수 있다”고 강조했다.

가짜 IT 인력 대응 전략

아마존(Amazon)은 AI 기반 도구와 인적 검토를 병행해 의심스러운 연락처 정보와 허위 학력, 가짜 기업 이력을 식별하고 있다. 또한 보안팀은 수상한 링크드인 프로필을 표시하고, 대면 면접과 사무실 출근을 강화하며, 컴퓨터 사용 패턴과 업무 품질을 모니터링하고 물리적 토큰 기반 인증을 적용하고 있다.

스티브 슈미트는 포츈 인터뷰를 통해 IT와 HR 부서 간 긴밀한 협력이 문제 해결의 핵심이라고 강조했다. 그는 “문제를 초기에 발견하는 것이 HR 조직 입장에서도 훨씬 비용 효율적”이라고 밝혔다.

센티넬원의 헤겔은 채용에 대한 접근 방식 자체를 바꿔야 한다고 지적했다. 그는 “채용을 단순 인사 절차가 아닌 접근 권한 통제 문제로 봐야 한다”라며 “신원을 한 번 확인하는 체크리스트로 끝내지 말고, 원격 채용을 특권 접근 권한 부여처럼 다뤄야 한다”고 설명했다.

에너지솔루션의 웨이송은 경험을 바탕으로 채용 시스템과 내부 프로세스 전반에 걸쳐 대대적인 변화를 도입했다.

채용 공고 단계부터 기술 직무 지원자가 요구사항과 책임을 명확히 이해하도록 모든 문서에 이를 명시했다. 웨이송은 “특히 ‘완전 원격 근무’라는 표현을 제거한 이후, 사기 시도와 해외 지원이 눈에 띄게 줄었다”고 말했다.

이어 “제로 트러스트 방식이 이상적이긴 하지만 채용 과정 자체를 저해하거나 정상 지원자를 위축시켜서는 안 된다”라며 “자동화된 사기 지원자가 애초에 채용 파이프라인에 들어오지 못하도록 충분한 대응책을 마련해야 한다”고 강조했다.

지원자 급증 문제를 해결하기 위해 에너지솔루션은 채용 공고에 강력한 CAPTCHA를 적용하고, 직원 추천 보너스를 통해 내부 네트워크 기반 채용을 확대했으며, 신규 입사자에게는 90일 성과 검증 기간을 운영하고 있다.

채용 심사 과정에서는 전화 대신 영상 면접을 실시하고, 실시간 과제를 위해 화면 공유를 요구한다. 또한 면접 이후 보고서를 통해 지원자의 실제 위치를 검증하며, 미국 외 지역에서 접속할 경우 ‘옐로/레드 플래그’로 분류한다.

지원자는 근무할 사무실을 직접 선택해야 하며, 면접 과정에서 AI 사용 시 탈락될 수 있다는 점에도 동의해야 한다.

경력 및 추천서 검증을 위해 최소 2명의 추천인을 요구하고, 그중 1명은 이전 상사 또는 관리자여야 한다. 과거 근무 이력과 이전 회사도 확인하며, 자택 주소 제출도 의무화했다.

접근 권한 통제를 위해 신규 직무가 민감 정보에 대한 고급 접근 권한을 포함하는지 여부를 사전 문서에서 확인하도록 했다.

입사 첫날에는 반드시 사무실에 출근해 장비를 수령하고 온보딩 교육을 받아야 하며, 모든 직무는 초기에는 온사이트 근무가 원칙이다. 이후 성과가 검증된 경우에만 하이브리드 근무가 허용된다.

웨이송은 “이 문제를 해결하기 위해서는 채용 프로세스를 재점검하고 HR과 긴밀히 협력하며, 각 대응 조치의 효과를 지속적으로 점검해야 한다”고 강조했다. 이어 “채용 시스템 자체가 잘못된 것이 아니라, 신뢰를 단계적으로 구축하는 방식으로 접근해야 한다”고 덧붙였다.
dl-ciokorea@foundryco.com

Antes de ontemStream principal
  • ✇Security | CIO
  • AI is spreading decision-making, but not accountability
    On a holiday weekend, when most of a company is offline, a critical system fails. An AI-driven workflow stalls, or worse, produces flawed decisions at scale that misprice products or expose sensitive data. In that moment, organizational theory disappears and the question of who’s responsible is immediately raised. As AI moves from experimentation into production, accountability is no longer a technical concern, it’s an executive one. And while governance frameworks sugg
     

AI is spreading decision-making, but not accountability

6 de Maio de 2026, 07:00

On a holiday weekend, when most of a company is offline, a critical system fails. An AI-driven workflow stalls, or worse, produces flawed decisions at scale that misprice products or expose sensitive data. In that moment, organizational theory disappears and the question of who’s responsible is immediately raised.

As AI moves from experimentation into production, accountability is no longer a technical concern, it’s an executive one. And while governance frameworks suggest responsibility is shared across legal, risk, IT, and business teams, courts may ultimately find it far less evenly distributed when something goes wrong.

AI, after all, may diffuse decision-making, but not legal liability.

AI doesn’t show up in court — people do

Jessica Eaves Mathews, an AI and intellectual property attorney and founder of Leverage Legal Group, understands that when an AI system influences a consequential decision, the algorithm isn’t what will show up in court. “It’ll be the humans who developed it, deployed it, or used it,” she says. For now, however, the deeper uncertainty is there’s very little case law to guide those decisions.

“We’re still in a phase where a lot of this is speculative,” says Mathews, comparing the moment to the early days of the internet, when courts were still figuring out how existing legal frameworks applied to new technologies. Regulators have signaled that responsibility can’t be outsourced to algorithms. But how liability will be apportioned across vendors, deployers, and executives remains unsettled — an uncertainty that’s unlikely to persist for long.

width="1240" height="827" sizes="auto, (max-width: 1240px) 100vw, 1240px">

Jessica Eaves Mathews, founder, Leverage Legal Group

LLG

“There are going to be companies that become the poster children for how not to do this,” she says. “The cases working their way through the system now are going to define how this plays out.”

In most scenarios, responsibility will attach first and foremost to the deploying organization, the enterprise that chose to implement the system. “Saying that we bought it from a vendor isn’t likely to be a defense,” she adds.

The underlying legal principle is familiar, even if the technology isn’t: liability follows the party best positioned to prevent harm. In an AI context, that tends to be the organization integrating the system into real-world decision-making, so what changes isn’t who’s accountable but how difficult it becomes to demonstrate appropriate safeguards were in place.

CIO as the system’s last line of defense

If legal accountability points to the enterprise, operational accountability often converges on the CIO. While CIOs don’t formally own AI in most organizations, they do own the systems, infrastructure, and data pipelines through which AI operates.

“Whether they like it or not, CIOs are now in the AI governance and risk oversight business,” says Chris Drumgoole, president of global infrastructure services at DXC Technology and former global CIO and CTO of GE.

The pattern is becoming familiar, and increasingly predictable. Business teams experiment with AI tools, often outside formal processes, and early results are promising. Adoption accelerates but controls lag. Then something breaks. “At that moment,” Drumgoole says, “everyone looks to the CIO first to fix it, then to explain how it happened.”

width="1240" height="827" sizes="auto, (max-width: 1240px) 100vw, 1240px">

Chris Drumgoole, president, global infrastructure services, DXC Technology

DXC

The dynamic is intensified by the rise of shadow AI. Unlike earlier forms of shadow IT, the risks here aren’t limited to cost or inefficiency. They extend to things like data leakage, regulatory exposure, and reputational damage.

“Everyone is an expert now,” Drumgoole says. “The tools are accessible, and the speed to proof of concept is measured in minutes.” For CIOs, this creates a structural asymmetry. They’re accountable for systems they don’t fully control, and increasingly for decisions they didn’t directly authorize.

In practice, that makes the CIO the enterprise’s last line of defense, not because governance models assign that role, but because operational reality does.

The illusion of distributed accountability

Most organizations, however, aren’t building governance structures around a single accountable executive. Instead, they’re constructing distributed models that reflect the cross-functional nature of AI.

width="1240" height="827" sizes="auto, (max-width: 1240px) 100vw, 1240px">

Ojas Rege, SVP and GM, privacy and data governance, OneTrust

OneTrust

Ojas Rege, SVP and GM of privacy and data governance at OneTrust, sees this distribution as unavoidable, but also potentially misleading. “AI governance spans legal, compliance, risk, IT, and the business,” he says. “No single function can manage it end to end.”

But that doesn’t mean accountability is shared in the same way. In Rege’s view, responsibility for outcomes remains firmly with the business. “You still keep the owners of the business accountable for the outcomes,” he says. “If those outcomes rely on AI systems, they have to figure out how to own that.”

In practice, however, governance is fragmented. Legal teams interpret regulatory exposure, risk and compliance define frameworks, and IT secures and operates systems. The result is a model in which responsibility appears distributed while accountability, when tested, is not — and it often compresses to a single point of failure. “AI doesn’t replace responsibility,” says Simon Elcham, co-founder and CAIO at payment fraud platform Trustpair. “It increases the number of points where things can go wrong.”

width="1240" height="827" sizes="auto, (max-width: 1240px) 100vw, 1240px">

Simon Elcham, CAIO, Trustpair

Trustpair

And those points are multiplying. Beyond traditional concerns such as security and privacy, enterprises must now manage algorithmic bias and discrimination, intellectual property infringement, trade secret exposure, and limited explainability of model outputs.

Each risk category may fall under a different function, but when they intersect, as they often do in AI systems, ownership becomes blurred. Mathews frames the issue more starkly in that accountability ultimately rests with whoever could have prevented the harm. The difficulty in AI systems is that multiple actors may plausibly claim, or deny, that role. So the result is a governance model that’s distributed by design, but not always coherent in execution.

The emergence and limits of the CAIO

To address this ambiguity, some organizations are beginning to formalize AI accountability through new leadership roles. The CAIO is one attempt to centralize oversight without constraining innovation.

At Hi Marley, the conversational platform for the P&C insurance industry, CTO Jonathan Tushman recently expanded his role to include CAIO responsibilities, formalizing what he describes as executive accountability for AI infrastructure and governance. In his view, effective AI governance depends on structured separation. “AI Ops owns how we build and run AI internally,” he says. “But AI in the product belongs to the CTO and product leadership, and compliance and legal act as independent checks and balances.”

The intention isn’t to eliminate tension, but to institutionalize it. “You need people pushing AI forward and people holding it back,” says Tushman. “The value is in that tension.”

width="1240" height="827" sizes="auto, (max-width: 1240px) 100vw, 1240px">

Jonathan Tushman, CTO, Hi Marley

Hi Marley

This reflects a broader shift in enterprise governance away from centralized control and toward managed friction between competing priorities — speed versus safety, innovation versus compliance. Yet even this model has limits.

When disagreements inevitably arise, someone must decide whether to proceed, pause, or reverse course. “In most organizations, that decision escalates often to the CEO or CFO,” says Tushman.

The CAIO, in other words, may coordinate accountability. But ultimate responsibility still sits at the top and can’t be delegated.

The widening gap between deployment and governance

If organizational models for AI accountability are still evolving, the gap between deployment and governance is already widening. “Companies are deploying AI at production speed, but governing at committee speed,” Mathews says. “That’s where the risk lives.”

Consequences are beginning to surface as a result. Many organizations lack even a basic inventory of AI systems in use across the enterprise. Shadow AI further complicates visibility, as employees adopt tools independently, often without understanding the implications.

The risks are both immediate and systemic. Employees may input sensitive corporate data into public AI platforms, inadvertently exposing trade secrets. AI-generated content may infringe on copyrighted material, and decision systems may produce biased or discriminatory outcomes that trigger regulatory scrutiny.

At the same time, regulatory expectations are rising, even in the absence of clear legal precedent. That combination — rapid deployment, limited governance, and legal uncertainty — makes it likely that a small number of high-profile cases will shape the future of AI accountability, as Mathews describes.

Where the buck stops

For all the complexity surrounding AI governance, one pattern is becoming clear. Responsibility may be distributed, authority may be shared, and new roles may emerge to coordinate oversight, but accountability doesn’t remain diffused indefinitely.

When systems fail, or when regulators intervene, it often points at enterprise leadership, and, in operational terms, to the executives closest to the systems in question. AI may decentralize how decisions are made, obscure the pathways through which those decisions emerge, and challenge traditional notions of control, but what it doesn’t do is eliminate responsibility. If anything, it magnifies it.

AI accountability is a familiar problem, refracted through a more complex system. The difference is the system is moving faster, and the cost of getting it wrong is increasing.

  • ✇Security | CIO
  • The fake IT worker problem CIOs can’t ignore
    Hiring fake IT workers has been a growing problem in recent years — but it’s often a problem very few want to admit to. From Fortune 500 companies down to smaller organizations, remote hiring practices have been exploited to grant trusted access to individuals who are not who they claim to be creating an insider threat risk. Estimates suggest there are thousands of fake IT workers operating across the US who are in a position to steal information, IP and data, outsource
     

The fake IT worker problem CIOs can’t ignore

5 de Maio de 2026, 07:01

Hiring fake IT workers has been a growing problem in recent years — but it’s often a problem very few want to admit to. From Fortune 500 companies down to smaller organizations, remote hiring practices have been exploited to grant trusted access to individuals who are not who they claim to be creating an insider threat risk.

Estimates suggest there are thousands of fake IT workers operating across the US who are in a position to steal information, IP and data, outsource work offshore, carry out sabotage, or funnel money to foreign governments.

Amazon has identified and blocked more than 1,800 attempts by North Korea to secure IT roles — and the numbers are rising, according to its chief security officer, Steve Schmidt.

In some cases, individuals impersonate US employees for personal gain; in others, state-based operatives such as those from North Korean pose as IT workers for state financial gain and other nefarious purposes.

AI is now enabling deepfakes, more convincing video interviews, and rapid identity cycling.

Adversary tactics are also shifting, from fabricating profiles to purchasing legitimate American identities, Schmidt has warned.

“This is not a ‘recruiting scam’ in the traditional sense. It’s an insider-risk problem, where the adversary’s first move is to get hired,” says Tom Hegel, distinguished threat researcher at SentinelOne.

CIOs, CISOs, and other IT leaders need to be continually on guard against fake and fraudulent IT workers, but organizations can fall victim without realizing it.

How fake hires get through

There’s no single point of failure in the recruitment process. Fake and fraudulent IT workers conceal their identity, falsify their skills and experience, and move through interview and screening processes undetected.

SentinelOne has tracked roughly 360 fake personas and more than 1,000 job applications linked to North Korean IT worker operations, including attempts to apply for roles within the company itself.

According to Hegel, adversaries are increasingly deploying social engineering tactics and identity obfuscation at scale, and the hiring process is a prime entry point.

Synthetic or stolen identities are used to create resumes and online profiles; interviews are passed with the help of scripts, stand-ins, or AI-assisted responses; and background checks confirm only what’s presented to them.

“Fake job seekers now leverage AI tools to mimic legitimate candidates, creating synthetic identities that pass initial background checks, falsifying employment histories and even responding convincingly in interviews using real-time AI assistance,” Hegel says.

Flashpoint investigations have found malware-infected hosts containing HR and job-board logins, browser histories showing Google-translated coaching notes, remote-access “laptop farms” used to control corporate devices from overseas, and shell companies to prove reference checks for fabricated resumes.

Once they’re hired, credentials are issued, equipment is shipped, and access is granted — and they become a trusted insider. “The long-term risk isn’t just hiring a fake employee — it’s unknowingly opening your systems and sensitive data to malicious access,” he says.

What to do if you suspect a fake IT worker

When a CIO suspects a fake IT worker, next steps are important as the issue shifts from recruitment to insider risk management.

During his time at MongoDB, George Gerchow, IANS faculty advisor and Bedrock Data CSO, oversaw the investigation after the company detected it had unknowingly hired a North Korean IT worker.

It was first discovered after alerts that an individual was attempting to uninstall endpoint protections, including CrowdStrike Overwatch. “Overwatch then detected the laptop communicating with a North Korean IP address,” says Gerchow.

“That combination of tool tampering plus DPRK-linked traffic immediately signaled that this was not a typical new hire,” he tells CIO.

Mongo realized the fake worker used a stolen identity, paired with AI-generated resume content and scripted interview responses, to evade background checks that verify only the information provided and do not detect fraud.


It highlights a gap in many background checks. “They don’t detect fabricated work histories, synthetic identities, or recycled developer profiles, which is how this individual passed screening and interviews without raising formal flags,” he says.

The subsequent investigation found attempts to disable security tooling, establish persistence on the device, and probe for elevated access.

“Had they remained undetected, their access would have eventually expanded into our FedRAMP environment, which makes these fraud techniques especially high-risk,” Gerchow adds.

After the discovery, several yellow flags became obvious such as poor video quality and unclear visuals during interviews, a noticeably inconsistent accent between calls, and scattered interview feedback with no centralized review.

Another tell was a last-minute change to the laptop shipping address. “That’s a common shadow-worker tactic,” notes Gerchow.

With hindsight, Gerchow joined the dots and it became clear how the person had made it through to employment because any irregularities were treated in isolation.

“None of these individually would prevent a hire. However, because no one was responsible for aggregating subtle anomalies, the pattern wasn’t recognized until the endpoint alert fired,” he says.

When they were discovered, the team quickly isolated the device, revoked all credentials, conducted a full forensic investigation, and notified federal authorities. “We verified there was no data exfiltration or lateral movement,” he says.

The mitigation steps introduced included strengthening identity fraud screening in the hiring process, assigning a Yellow Flag owner to connect early signals, and enforcing zero access until trust is earned for new hires,


Gerchow also believes that behavioral telemetry post-hire is necessary, because behavior, not credentials, reveals impostors.

Mongo recommends organizations designate a reviewer in Security or HR to identify inconsistencies in the hiring process, such as poor video quality. “Also watch for AI-generated LinkedIn profiles, mismatched resumes and questionable changes in laptop shipping addresses,” he says.

“Use panel interviews and project-based evaluations to identify candidates who recycle stolen or fake developer identities, and start new hires without access to sensitive data or production environments,” he advises.

Then employ alerts if security agents (such IAM, EDR, VPN) are disabled before a new hire logs in, and test detection, escalation, and device recovery by simulating the hiring of a fake developer.

“And look for off-hours access, broad internal search activity and large-scale cloning of documents or code repositories,” he adds.

What IT leaders see on the inside

The problem of employment fraud is only expected to worsen, with Gartner predicting that one in four candidate profiles worldwide will be fake by 2028.

“The rise of fake and fraudulent job applicants has become an epidemic across organizations,” says David Weisong, CIO of Energy Solutions.

Weisong says attackers consistently target high-access technical roles such as DevOps, systems administrators, data engineers, and database administrators, where successful hires can gain deep visibility and control over core systems.

“These are the roles with the keys to the castle,” Weisong says. “If you’re trying to gain access, they’re far more valuable than a standard developer position.”

Operating in a regulated energy market, Energy Solutions is contractually required to employ a US-based workforce and keep data within US jurisdiction.

Weisong has first-hand experience with detecting fake IT workers and wants to share his advice with other IT leaders. One of the earliest warning signs was a sudden, abnormal surge in applications — hundreds arriving within hours, far out of proportion to the company’s brand profile, pointing to automated or coordinated activity.

During the interview stage, identity switching was observed. “We saw cases where one person passed the phone screen, a different person showed up on Zoom, and sometimes a third appeared later — all under the same name and resume,” Weisong says.

Part of the problem is that standard hiring practices validate information and skills in isolation. “Traditional background checks only verify the information provided and do not detect fraud,” Weisong also notes.

The uncomfortable reality for some CIOs is that the work may be completed to a high standard and detection comes from signals, not performance.

However, fake IT workers create business and compliance risk as much as security risk, exposing organizations to contractual breaches, regulatory consequences, and loss of client trust — particularly in regulated industries.

Weisong says fake IT workers create business and compliance risk as much as security risk, exposing organizations in regulated industries to contractual breaches, regulatory scrutiny, and loss of client trust.

Combating the problem of fake IT workers

Amazon is using AI-based tools with human oversight to identify unusual contact information, as well as fake academic institutions and companies in resumes, according to Schmidt. Security teams will flag LinkedIn profiles that look suspicious, require more in-person interviews and in-office attendance, monitor computer usage and quality of work, and authenticate with a physical token.

He has also said that IT and HR need to collaborate on hiring to combat the problem.

“It’s actually a lot cheaper for the HR organization if we discover the problem up front,” Amazon’s Schmidt told Fortune.

The shift required, says SentinelOne’s Hegel, is treating hiring decisions as an access control problem rather than a recruitment task. “Stop treating identity as a one-time HR checkbox and start treating remote hiring like you would grant privileged access,” he says.

In the wake of his experience, Weisong instituted a raft of changes to its applicant tracking system and across the organization’s internal systems and processes.

When advertising for positions, they make it clear that candidates applying for technical positions understand the expectations and consequences outlined in all written communication. “Additionally, removing the term ‘fully remote’ from our hiring practices has significantly reduced opportunities for fraud and for applicants applying from outside the US,” he says.

“While a ‘zero-trust’ approach would be ideal for all hiring, we cannot allow it to impede the process or discourage legitimate candidates from applying. Instead, we need sufficient countermeasures to prevent automated and fraudulent applicants from reaching the pipeline in the first place,” he adds.

To control the large volume of applications, many of which are bots, Energy Solutions job listings now have strict CAPTCHA settings, referral bonuses help draw on employee networks, and there’s a 90-day satisfactory performance review for new hires.

During the screening process, interviews are conducted via video not phone, and applicants must share their screen for live challenges. A post-video interview report allows them to verify the exact location of applicants after screening and interview meetings. If a candidate is outside the US, it’s treated as a Yellow/Red flag.

Applicants must select which office they want to work from and they must acknowledge they understand use of AI during interviews will result in disqualification.

To verify references and employment history, they require two references, with one a former supervisor or manager. Employment history is checked, including previous employers, and full home address must be provided.

To guard access, a question has been added to the job kick-off form that indicates whether a new role will have elevated access to confidential or sensitive information.

The first day on the job requires new hires to come into an office to pick up equipment and undertake training and onboarding. All roles must be onsite, with the option to go hybrid after satisfactory performance.

Combating the problem, says Weisong, requires reviewing hiring processes, partnering closely with HR, and monitoring the effectiveness of each countermeasure. For CIOs, the lesson is not that hiring is broken, but that trust must be earned progressively.

NIST Cybersecurity Framework for UK SMEs: A Practical Guide to Identify, Protect, Detect, Respond, and Recover

1 de Maio de 2026, 07:02

NIST Cybersecurity Framework for UK SMEs: A Practical Guide to Identify, Protect, Detect, Respond, and Recover The NIST Cybersecurity Framework is a useful way to organise cybersecurity work around business risk. For UK SMEs, that matters because most teams do not have the time or budget to do everything at once. A framework gives you […]

The post NIST Cybersecurity Framework for UK SMEs: A Practical Guide to Identify, Protect, Detect, Respond, and Recover appeared first on Clear Path Security Ltd.

The post NIST Cybersecurity Framework for UK SMEs: A Practical Guide to Identify, Protect, Detect, Respond, and Recover appeared first on Security Boulevard.

  • ✇Security Boulevard
  • Threat modelling using STRIDE for system architects Clear Path Security Ltd
    Threat modelling using STRIDE for system architects Threat modelling is one of the most useful habits a system architect can build. Done well, it helps you spot design weaknesses before they become expensive problems to fix. Done badly, it turns into a long list of theoretical threats that nobody uses. STRIDE is a simple way […] The post Threat modelling using STRIDE for system architects appeared first on Clear Path Security Ltd. The post Threat modelling using STRIDE for system architects appe
     

Threat modelling using STRIDE for system architects

30 de Abril de 2026, 08:05

Threat modelling using STRIDE for system architects Threat modelling is one of the most useful habits a system architect can build. Done well, it helps you spot design weaknesses before they become expensive problems to fix. Done badly, it turns into a long list of theoretical threats that nobody uses. STRIDE is a simple way […]

The post Threat modelling using STRIDE for system architects appeared first on Clear Path Security Ltd.

The post Threat modelling using STRIDE for system architects appeared first on Security Boulevard.

  • ✇Security Boulevard
  • Protective Security in the NCSC CAF: A Practical Guide for UK SMEs Clear Path Security Ltd
    Protective security is one of those topics that can sound broader and more complex than it needs to be. For UK SMEs, the practical question is simple: what do you need to protect, how much protection is enough, and how do you make it work without creating unnecessary overhead? Within the NCSC Cyber Assessment Framework, […] The post Protective Security in the NCSC CAF: A Practical Guide for UK SMEs appeared first on Clear Path Security Ltd. The post Protective Security in the NCSC CAF: A Practic
     

Protective Security in the NCSC CAF: A Practical Guide for UK SMEs

29 de Abril de 2026, 08:22

Protective security is one of those topics that can sound broader and more complex than it needs to be. For UK SMEs, the practical question is simple: what do you need to protect, how much protection is enough, and how do you make it work without creating unnecessary overhead? Within the NCSC Cyber Assessment Framework, […]

The post Protective Security in the NCSC CAF: A Practical Guide for UK SMEs appeared first on Clear Path Security Ltd.

The post Protective Security in the NCSC CAF: A Practical Guide for UK SMEs appeared first on Security Boulevard.

  • ✇Security Boulevard
  • The $700 million question: How cyber risk became a market cap problem Shweta Dhole
    Cyber risk used to be the kind of problem you could delegate. Something for the CISO, the IT team, and maybe an external auditor to worry about once a year. That comfort zone is gone. In the last decade, a new reality has set in: a single cyber incident can erase hundreds of millions of […] The post The $700 million question: How cyber risk became a market cap problem first appeared on TrustCloud. The post The $700 million question: How cyber risk became a market cap problem appeared first on Se
     

The $700 million question: How cyber risk became a market cap problem

27 de Abril de 2026, 03:39

Cyber risk used to be the kind of problem you could delegate. Something for the CISO, the IT team, and maybe an external auditor to worry about once a year. That comfort zone is gone. In the last decade, a new reality has set in: a single cyber incident can erase hundreds of millions of […]

The post The $700 million question: How cyber risk became a market cap problem first appeared on TrustCloud.

The post The $700 million question: How cyber risk became a market cap problem appeared first on Security Boulevard.

  • ✇Security Boulevard
  • Safe vulnerability disclosure for UK SMEs: a practical guide Clear Path Security Ltd
    Safe vulnerability disclosure for UK SMEs: a practical guide For many UK SMEs, the idea of someone reporting a security weakness can feel unsettling at first. It may sound technical, formal, or even a little confrontational. In practice, safe vulnerability disclosure is simply a controlled way for people to tell you about a security issue […] The post Safe vulnerability disclosure for UK SMEs: a practical guide appeared first on Clear Path Security Ltd. The post Safe vulnerability disclosure for
     

Safe vulnerability disclosure for UK SMEs: a practical guide

27 de Abril de 2026, 03:29

Safe vulnerability disclosure for UK SMEs: a practical guide For many UK SMEs, the idea of someone reporting a security weakness can feel unsettling at first. It may sound technical, formal, or even a little confrontational. In practice, safe vulnerability disclosure is simply a controlled way for people to tell you about a security issue […]

The post Safe vulnerability disclosure for UK SMEs: a practical guide appeared first on Clear Path Security Ltd.

The post Safe vulnerability disclosure for UK SMEs: a practical guide appeared first on Security Boulevard.

Supplier assurance for UK SMEs: a practical guide to checking third parties without overcomplicating it

25 de Abril de 2026, 10:24

Supplier assurance for UK SMEs: a practical guide to checking third parties without overcomplicating it Most UK SMEs rely on suppliers in some way. That might be payroll software, a managed IT provider, a marketing agency, a logistics partner, or a cloud service that holds customer data. The more your business depends on third parties, […]

The post Supplier assurance for UK SMEs: a practical guide to checking third parties without overcomplicating it appeared first on Clear Path Security Ltd.

The post Supplier assurance for UK SMEs: a practical guide to checking third parties without overcomplicating it appeared first on Security Boulevard.

  • ✇Security | CIO
  • AI 책임 보장서 발 빼는 보험사들…불확실성에 시장 재편 조짐
    주요 보험사가 AI를 활용해 내부 프로세스를 운영하는 기업에 대한 사이버 보안 및 기타 보험 제공에서 점차 발을 빼고 있는 것으로 전해졌다. 보험 시장에서 고객의 AI 활용에 대한 일관된 대응 기준은 없지만, 많은 보험사가 사이버 보안 및 전문직 배상책임보험(E&O, Errors and Omissions 보험)에서 AI가 생성한 결과물과 관련된 청구에 대해 계약 체결을 조용히 거부하는 사례가 늘고 있다고 업계 관계자들은 전했다. 일부 보험사는 AI 관련 사고를 보장하기 위해 보험료를 크게 인상하는 방식으로 대응하고 있다. 보험사들과 협력하는 AI 개발 및 컨설팅 기업 코드스트랩의 CEO 코너 딕스는 “수십 개 보험사가 AI 관련 오류 보장에 대해 재검토에 나선 것으로 보인다”고 말했다. 이어 딕스는 “많은 보험사가 AI 결과물을 보장하는 데 부담을 느끼고 있다”며 “AI가 어떤 과정을 거쳐 결과를 도출했는지 추적하기 어
     

AI 책임 보장서 발 빼는 보험사들…불확실성에 시장 재편 조짐

23 de Abril de 2026, 23:08

주요 보험사가 AI를 활용해 내부 프로세스를 운영하는 기업에 대한 사이버 보안 및 기타 보험 제공에서 점차 발을 빼고 있는 것으로 전해졌다.

보험 시장에서 고객의 AI 활용에 대한 일관된 대응 기준은 없지만, 많은 보험사가 사이버 보안 및 전문직 배상책임보험(E&O, Errors and Omissions 보험)에서 AI가 생성한 결과물과 관련된 청구에 대해 계약 체결을 조용히 거부하는 사례가 늘고 있다고 업계 관계자들은 전했다. 일부 보험사는 AI 관련 사고를 보장하기 위해 보험료를 크게 인상하는 방식으로 대응하고 있다.

보험사들과 협력하는 AI 개발 및 컨설팅 기업 코드스트랩의 CEO 코너 딕스는 “수십 개 보험사가 AI 관련 오류 보장에 대해 재검토에 나선 것으로 보인다”고 말했다.

이어 딕스는 “많은 보험사가 AI 결과물을 보장하는 데 부담을 느끼고 있다”며 “AI가 어떤 과정을 거쳐 결과를 도출했는지 추적하기 어렵기 때문”이라고 설명했다.

그는 “이러한 흐름은 사이버 보안과 E&O 전반에서 보험 보장 범위를 제외하는 형태로 나타나고 있다”며 “최근 등장한 다양한 바이브 코딩 기반 솔루션과 AI 시스템에는 본질적인 리스크가 내재돼 있지만, 전체 처리 과정을 완전히 들여다보기 어렵다”고 지적했다.

보험업계의 이러한 우려는 2025년 11월 파이낸셜타임스 보도를 통해 처음 드러났다. 당시 AIG, 그레이트 아메리칸 인슈어런스 그룹(Great American), WR버클리(W.R. Berkley) 등 주요 보험사는 챗봇과 AI 에이전트 등 AI 도구와 관련된 책임을 제외한 보험 상품을 제공하기 위해 미국 규제 당국에 신청서를 제출했다. 당시에는 향후 AI 관련 오류를 제외하기 위한 사전 대응으로 해석됐다.

그러나 현재는 다수 보험사가 실제로 AI 오류를 보험 약관에서 제외하는 방향으로 정책을 구체화하고 있다고 딕스는 밝혔다. 그는 “일부 보험사는 AI로 인한 비즈니스 중단과 책임에 대한 보장을 제한하거나 아예 중단하는 움직임을 보이고 있다”고 덧붙였다. 이어 “아이러니하게도 많은 보험사가 내부적으로는 AI 활용을 확대하고 있다”고 말했다.

딕스가 이끄는 코드스트랩은 AI 보험 시장 확대에 이해관계가 있는 기업으로, 자사 AI 코딩 플랫폼을 ‘추적 가능하고 보험 적용이 가능한 기술’로 홍보하고 있다. 다만 업계 다른 관계자들 역시 유사한 보험사들의 움직임을 확인하고 있는 것으로 전해졌다.

보험사, AI 보장 제외 확대

글로벌 보험사 NSI 인슈어런스 그룹의 재무 부문 책임자 제이슨 비샤라는 “얼마나 많은 보험사가 AI 워크로드 보험을 거부할지는 아직 불확실하지만, 일부 보험사는 이미 AI로 인한 비즈니스 혼란을 보장 대상에서 제외하는 보험 상품을 출시하고 있다”고 말했다.

이어 “보험사들의 리스크 수용 범위는 지속적으로 변화하고 있다”며 “AI와 관련해서는 아예 리스크 대상에서 제외하고 보험 견적 자체를 거부하는 사례도 나오고 있다”고 설명했다.

비샤라는 일부 보험사가 AI 결과물에 대한 보장을 거부하는 반면, 다른 보험사는 증가한 리스크를 반영해 보험료를 인상하고 있다고 전했다. 구체적인 인상 폭은 공개되지 않았지만 상당한 수준이라는 설명이다.

그는 “모든 기업은 보험에 가입하고 있으며, 동시에 대부분의 기업이 일정 수준 AI를 활용하고 있다”며 “이러한 상황에서 보험 약관에 AI 관련 책임 제외 조항이 포함되고, 보험사들이 이를 회피하는 경향이 나타나고 있는가에 대한 질문에는 ‘그렇다’고 답할 수 있다”고 밝혔다.

또한 보험사들은 AI 공급업체와 AI를 활용하는 일반 기업을 구분해 대응하고 있다. 많은 경우 보험사는 AI 기업에 대한 보장을 아예 거부하는 반면, 기술을 사용하는 기업에는 일부 예외 조항을 두는 방식으로 접근하고 있다.

비샤라는 “AI 관련 기업이거나 순수 AI 기업이라면 현재로서는 보험 가입이 거절될 가능성이 높다”고 말했다.

최근 몇 달 사이 보험사들은 고객의 AI 활용 방식에 대해 보다 구체적인 질문을 던지고 있다. 이는 사고 발생 가능성에 따른 리스크를 정밀하게 평가하기 위한 조치다. 다만 이러한 심사 강화는 결과적으로 기업이 AI 워크로드에 대한 보험을 가입하기 어렵게 만드는 요인으로 작용할 전망이다.

비샤라는 “현재 AI를 활용하는 기업이라면 ‘AI 정책은 무엇인지, 어떤 절차를 따르는지, 실제 비즈니스에서 어떻게 활용하고 있는지’와 같은 질문을 받고 있다”며 “보험 인수 담당자들 역시 ‘기업이 AI를 어떻게 활용하고 있는지’에 대해 매우 상세하게 확인하고 있다”고 설명했다.

보험 적용 기준 ‘재편’ 단계

MSP 기업 엔소노의 보험 부문 CTO 필 카레키는 일부 보험사가 AI 결과물 보장에서 물러나는 움직임을 보이고 있지만, 이것이 업계 전반의 흐름인지에 대해서는 신중한 입장을 보였다. 그는 보험사들이 보장 방식을 지속적으로 실험해 왔다고 설명했다.

카레키는 “보험사들은 보장 여부를 판단할 때 엄격하게 통제된 AI 도입과 실험적 프로젝트를 구분하려 하고 있다”고 말했다.

이어 “현재 AI는 통제된 생성형 AI와 자율형 AI로 양분되는 양상을 보인다”며 “이제는 단순히 ‘AI를 사용하고 있는가’가 아니라 ‘얼마나 통제되고 있는가, 어떻게 관리하고 있으며 얼마나 안전하게 운영되는가’가 핵심 질문이 됐다”고 설명했다.

그는 보험사들이 AI 워크로드 보장이 수익성이 있는지 판단하려는 단계에 있다고 덧붙였다. 명확한 의사결정 범위 내에서 운영되는 통제형 AI는 상대적으로 보험 적용이 가능하지만, 모니터링이 어렵고 롤백이 불가능한 실험적 시스템은 보장이 어렵다는 분석이다.

카레키는 “이는 단순한 후퇴라기보다 재정립 과정에 가깝다”며 “보험사들은 특정 상품이 시장성이 있는지 확인하기 위해 보장을 확대했다가, 결과를 분석한 뒤 다시 진입 여부를 결정하는 전략을 반복하기도 한다”고 말했다.

또한 AI 시스템의 보험 가능 여부는 개별 고객 환경에 따라 달라질 수 있다고 설명했다. 그는 “보험사들은 기본적으로 보험 사업을 포기하려는 것이 아니라, 해당 영역이 수익성이 있는지 판단하려는 것”이라며 “각 계약마다 ‘AI를 어떤 용도로 사용하는지, 어떻게 통제하는지, 어떤 리스크가 발생하는지’와 같은 질문이 뒤따르고 있다”고 전했다.

AI 개발 및 컨설팅 기업 코드스트랩 CTO 도리안 스마일리는 현재 AI 시스템의 신뢰성이 낮다는 점에서 보험사의 우려가 합리적이라고 평가했다. 그는 “이론적으로 AI 모델은 동일한 입력에 동일한 결과를 내야 하지만, 실제로는 같은 입력에서도 전혀 다른 결과가 나올 수 있다”며 “결과의 정확성을 스스로 검증하지 못하는 경우도 많다”고 지적했다.

또한 “대부분의 AI 모델은 귀납적 추론 능력이 부족하고 자체 검증도 어렵다”며 “그럼에도 많은 기업이 수백 개의 자율형 에이전트를 디지털 직원처럼 운영하려는 것은 비현실적”이라고 평가했다. 이어 “새로운 정보를 학습하지 못하고, 정보를 안정적으로 조회하지 못하며, 자신의 결과를 검증할 수 없는 사람을 채용하지는 않을 것”이라고 비판했다.

한편 NSI 인슈어런스 그룹의 제이슨 비샤라는 AI 보험 가입을 고려하는 IT 및 비즈니스 리더들에게 투명성을 강조했다. 그는 “AI 활용 방식을 충분히 공개하지 않을 경우, 사고 발생 시 보험금 지급이 거절될 수 있다”며 “보험사가 해당 리스크를 사전에 인지하지 못했다는 이유로 보장을 거부할 가능성이 있다”고 경고했다.
dl-ciokorea@foundryco.com

  • ✇Security Boulevard
  • Supply Chain Resilience for UK SMEs: Practical Steps to Reduce Third-Party Risk Clear Path Security Ltd
    For many UK SMEs, supply chain resilience is not a specialist security project. It is a business continuity issue. If a key supplier cannot deliver, a software provider has an outage, or a partner mishandles data, the impact can show up quickly in customer service, cash flow, and reputation. The good news is that you […] The post Supply Chain Resilience for UK SMEs: Practical Steps to Reduce Third-Party Risk appeared first on Clear Path Security Ltd. The post Supply Chain Resilience for UK SMEs:
     

Supply Chain Resilience for UK SMEs: Practical Steps to Reduce Third-Party Risk

23 de Abril de 2026, 07:26

For many UK SMEs, supply chain resilience is not a specialist security project. It is a business continuity issue. If a key supplier cannot deliver, a software provider has an outage, or a partner mishandles data, the impact can show up quickly in customer service, cash flow, and reputation. The good news is that you […]

The post Supply Chain Resilience for UK SMEs: Practical Steps to Reduce Third-Party Risk appeared first on Clear Path Security Ltd.

The post Supply Chain Resilience for UK SMEs: Practical Steps to Reduce Third-Party Risk appeared first on Security Boulevard.

  • ✇Security | CIO
  • How the EU’s NIS2 directive is changing how CIOs think about digital infrastructure
    In conversations I’ve had with CIOs over the past year, there’s been a noticeable shift in how NIS2 (Network and Information Security Directive 2) is being discussed. It used to be filed away as another regulatory hurdle to clear, but now it’s prompting CIOs and their teams to think a little deeper about how well they understand the systems they depend on. For a long time, risk has been largely framed within the boundaries of the organization — something that could be mana
     

How the EU’s NIS2 directive is changing how CIOs think about digital infrastructure

23 de Abril de 2026, 08:00

In conversations I’ve had with CIOs over the past year, there’s been a noticeable shift in how NIS2 (Network and Information Security Directive 2) is being discussed. It used to be filed away as another regulatory hurdle to clear, but now it’s prompting CIOs and their teams to think a little deeper about how well they understand the systems they depend on. For a long time, risk has been largely framed within the boundaries of the organization — something that could be managed through internal controls, policies and audits. But that no longer reflects how digital services are built or delivered. Most organizations I encounter rely on a web of providers spanning cloud platforms, data centers, network operators and software vendors, all working together to create a “patchwork” ecosystem. NIS2 is different because it acknowledges that reality and, in doing so, it’s forcing a broader and sometimes more uncomfortable reassessment of where risk really sits.

What stands out to me is that NIS2 doesn’t just focus on individual accountability, but on the very definition of resilience itself. It recognizes that disruption rarely originates within a single process, or even a single organization. More often, it emerges from the connections between them; from unseen dependencies, indirect relationships and assumptions about how systems will behave under pressure. That’s novel, because it moves the conversation away from whether individual systems are secure, and toward whether the overall architecture those systems sit within can continue to function when something inevitably goes wrong. In that sense, NIS2 is less about tightening cybersecurity controls and more about encouraging a different way of thinking, where resilience is shaped as much by how infrastructure is designed and connected as it is by how it is protected.

NIS2 expands the definition of risk beyond the enterprise

One of the most immediate impacts I’m seeing from NIS2 is how it challenges long-held assumptions about control. Speak to any CIO, and they’ll usually talk about securing what sits within their own environments — their applications, services and data. But in practice, very little of today’s digital estate is fully owned because it’s so distributed among third parties with countless links and dependencies. Virtually all business services depend on layers of external providers, each with its own dependencies, architectures and risk profiles. According to the World Economic Forum, the top supply chain risk in 2026 is the inheritance risk — the inability to ensure the integrity of third-party software, hardware or services. NIS2 brings that into sharp focus by extending accountability beyond direct suppliers to include the wider ecosystem that supports them. In essence, it prompts businesses to shift from asking “are we secure?” to “how secure is everything we rely on to operate?”

That’s quite a challenge, because it’s not enough for businesses to simply know their suppliers — they need to understand how deeply interconnected those relationships are. In many cases, the real exposure sits several steps removed, in the providers behind your providers or in shared infrastructure that underpins multiple services at once. The “uncomfortable reassessment” I mentioned earlier is the squaring of this circle — how many organizations have full visibility into that sprawling landscape, let alone the means to control it?

NIS2 is compelling organizations to map dependencies more rigorously, to ask harder questions of their partners and network infrastructure, and to recognize that resilience is only as strong as the most fragile link in the chain. The WEF shows that in 2026, only 33% of organizations map their entire IT supply chain to gain this visibility. And even then, the added risk of unknown service providers, such as is the case when suing the public Internet, where data pathways are neither visible nor controllable, is difficult to quantify.

Compliance is the trigger, but architecture is the challenge

What I find interesting about NIS2 is that it goes deeper than compliance — it’s trying to trigger a shift in culture. It’s relatively straightforward to introduce new policies, expand reporting requirements or formalize supplier assessments. But what happens when those requirements collide with the reality of how modern IT environments are built? Many organizations simply don’t have a clear, end-to-end view of how their services are delivered, how data flows between providers or how incidents might spread like wildfire across the ecosystem they depend on. NIS2 asks CIOs to look beyond governance frameworks and examine whether their operating models support the level of oversight and responsiveness the directive expects.

And that is where the architecture question becomes essential. It’s one thing to require suppliers to report incidents or meet certain security standards; it’s another thing entirely to ensure that the underlying infrastructure is designed to absorb disruption without cascading failure. In my experience, this is where many organizations begin to realize that resilience cannot be layered on afterwards. It must be built into how systems are structured, how dependencies are managed and how connectivity is established between environments. NIS2 may define what needs to be done, but it doesn’t prescribe how to do it. That responsibility sits with CIOs, who now have to translate regulatory intent into practical design decisions about where workloads run, how services interconnect and how failure is contained when it occurs.

Infrastructure design is now resilience design

What this ultimately leads to is a big infrastructure rethink. I’m privileged to have had some interesting discussions with CIOs and other executives about this very topic, so I know that resilience is beginning to be understood as more than a set of security controls. Connectivity is now at the heart of resilience, and in that sense, NIS2 has succeeded in getting organizations to think differently about what resilience really means. If a service depends on a single cloud region, a single network path or a tightly coupled set of providers, then no amount of policy or monitoring will prevent disruption when one of those elements fails. I’m pleased to see organizations starting to question these assumptions — not just asking whether systems are secure, but whether they are structured in a way that allows them to continue operating under stress. That shift in thinking does away with the abstract theory of resilience and defines it as something that can be designed and architected.

From a connectivity perspective, this means building in diversity at every level. Distributing workloads across geographically separate locations, establishing multiple, independent network paths and avoiding unnecessary concentration of critical services all contribute to a more resilient architecture. Interconnection plays a starring role here as the mechanism that allows different parts of the digital ecosystem to communicate in controlled, redundant and predictable ways. When designed properly, this kind of architecture limits the blast radius of any single point of failure and makes it easier to maintain service continuity even when parts of the system are down or under strain. The real takeaway here is that resilience is not something any single organization can achieve in isolation. It emerges from the collective design of the entire ecosystem, where each participant contributes to the overall stability of the services they all depend on.

When regulatory pressure gives way to strategic opportunity

The building blocks are already there. Practices like supplier due diligence, security certifications and business continuity planning are not new. What NIS2 does is raise the bar on how consistently and how deeply they are applied. It also brings a level of structure to conversations that were previously fragmented, particularly when it comes to expectations between partners. And therein lies the strategic upside. Organizations that can clearly demonstrate how they manage risk across their supply chains, how they design for resilience and how they respond to disruption are in a stronger position, not just from a regulatory standpoint, but in how they engage with customers and partners. In some sectors, we’re already seeing this play out through increased requests for transparency, self-assessments and proof of compliance. That trend is only going to accelerate. For CIOs, it’s a golden opportunity to move beyond a defensive posture and position resilience as a key competitive differentiator. It becomes a way to build trust, strengthen relationships and support more sustainable growth, rather than simply a requirement to satisfy regulators.

NIS2 may be the catalyst, but the underlying change runs deeper. It’s pushing CIOs to think beyond compliance and toward a more structural understanding of risk that reflects how digital services operate today.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

  • ✇Security | CIO
  • Insurance carriers quietly back away from covering AI outputs
    Several major insurance carriers have begun to back away from providing cybersecurity and other insurance to companies using AI to run internal processes, insiders say. While there’s no standard response to customer use of AI in the insurance market, many carriers are now quietly declining to write policies for claims related to AI-generated outputs in cybersecurity and errors and omissions (E&O) coverage, these observers say. Other insurance carriers are jacking up
     

Insurance carriers quietly back away from covering AI outputs

20 de Abril de 2026, 06:00

Several major insurance carriers have begun to back away from providing cybersecurity and other insurance to companies using AI to run internal processes, insiders say.

While there’s no standard response to customer use of AI in the insurance market, many carriers are now quietly declining to write policies for claims related to AI-generated outputs in cybersecurity and errors and omissions (E&O) coverage, these observers say. Other insurance carriers are jacking up prices to cover AI-related claims, they say.

Dozens of insurance carriers appear to be rethinking coverage for mistakes related to AI, says Connor Deeks, CEO of Codestrap, an AI development and consulting firm that works with insurance firms.

Many insurance companies aren’t comfortable with covering AI outputs because they can’t track the reasoning path the AI took to come up with a result, he says.

“That’s playing out downstream with insurance companies basically carving out coverage, whether that’s across cybersecurity or E&O,” he says. “All of these vibe-coded solutions and these AI systems that people have constructed have inherent risk baked into the cake now, and you can’t actually see the full process.”

The insurance carrier concerns about AI workloads first surfaced in November 2025, when Financial Times reported that three major carriers, AIG, Great American, and W.R. Berkley, filed requests with US regulators to offer insurance policies that exclude liabilities tied to AI tools such as chatbots and agents. At the time, those requests appeared to be preemptive moves to be allowed to exclude AI mistakes sometime in the future.

But now, many carriers seem to be moving forward with plans to exclude AI mistakes from policies, Deeks says. Several carriers he’s been in contact with are moving to limit or end coverage for AI-related business disruptions and liabilities, he adds. The irony is that many insurance carriers are embracing AI for their own internal purposes.

Deeks’ company has a vested interest in AI insurance coverage — Codestrap markets its AI coding platform as traceable and therefore insurable — but other industry insiders have also seen similar carrier decisions.

Carriers find exclusions

It’s still unclear how many carriers will refuse to insure AI workloads, but several carriers are now writing insurance policies that exempt coverage for AI-related business chaos, says Jason Bishara, financial practice leader at global carrier NSI Insurance Group.

“The risk appetite is changing among the carriers, and it’s always constantly evolving,” he says. “With regard to AI, there are carriers that are just removing it from their risk appetite and declining to quote altogether.”

While some carriers have declined to cover AI outputs, others are building in rate hikes to cover the increased risk, Bishara says. While he doesn’t have numbers on the extent of the rate hikes, they are significant, he adds.

“Every business has insurance, and every business now is using AI to some extent,” he adds. “Are you seeing those liabilities and exclusions within these policies and an aversion to it from the carriers? The answer is yes.”

Carriers are also treating AI vendors differently than AI users, he says. In many cases, carriers are declining to cover AI vendors altogether, while they carve out exceptions in policies against covering AI at companies using the technology.

“If you’re an AI-related company or specifically an AI company, there’s a good chance that you’ll get a declination at this point,” he adds.

In recent months, many carriers have been asking detailed questions about how customers are using AI to better understand the risk of insuring potential mishaps, he says. Ultimately, this increased scrutiny will make it more difficult for companies to buy insurance for AI workloads.

“For everybody leveraging AI right now, you’re seeing questions like, ‘What are your AI policies? What are your procedures? How are you leveraging AI within your business?’” Bishara adds. “We’re getting a lot of questions from the underwriters on, ‘How do you leverage AI within your business?’”

Coverage in flux

Phil Karecki, CTO for the insurance sector at managed services provider Ensono, also sees some carriers backing away from covering AI outputs, although he’s not sure whether it’s a major trend. Insurance carriers continuously experiment with how to provide coverage, he notes.

Carriers have tried to separate tightly governed AI deployments from more experimental projects when determining whether to provide coverage, he says.

“You’ve got this bifurcation of AI, the governed generative and the autonomous pieces,” he says. “It’s no longer, ‘Are you using AI?’ It’s asking, ‘Are you using governed AI? How are you governing it? How are you keeping it safe and secure?’”

Carriers have been trying to determine whether covering AI workloads can be profitable for them, Karecki adds. Governed AI tools operating in a bounded decision-making process will be more insurable, while experimental AI systems with no monitoring and no easy rollback will be difficult to cover, he notes.

“There’s a repositioning versus a pullback, and that’s very common to the industry, and they will at times open up coverage just to see if it’s this type of insurance that will sell,” he says. “They will assess the results and what needs to change so they can decide whether to re-enter this marketplace or abandon it completely.”

In some cases, whether an AI system is insurable may come down to circumstances at individual insurance customers. Carriers in general don’t want to get out of the business of providing insurance, Karecki says.

“What they’re working for right now is, ‘How do I make this profitable, and is this sector insurable?’” he says. “They make those decisions on every application regardless, but now, depending upon what they’re being asked to insure, the questions will follow. ‘What are you using AI for? How are you governing it? What risks does that introduce?’”

It makes sense that some carriers have begun to question whether to cover AI outputs, given the current level of unreliability of most AI systems, says Dorian Smiley, CTO at Codestrap.

“The math says these models should be deterministic, like given the same input, you should get the same output,” he says. “But you can get very different output from the same input, and they can’t know if the answer that they’re giving you is actually correct.”

In most cases, AI models lack inductive reason and can’t review their own work, but many organizations are talking about deploying hundreds of autonomous agents and treating them like digital employees, he notes.

“The idea that these agents are going to become employees, autonomous people working in your organization, is insane,” he says. “You would never hire a person that can’t learn new information, can’t reliably retrieve information, or check their own work.”

NSI’s Bishara has advice for IT and business leaders looking for insurance coverage for their AI workloads: Be honest about how they’re using AI. If they try to hide their AI risks, they risk having their claims rejected when something goes wrong, he says.

“If you don’t fully disclose these things appropriately in the way in which you’re functioning and operating, it could be utilized as an excuse to deny a claim at a later date,” he says. “You don’t want a carrier to come back and say, ‘We didn’t underwrite to that risk. We asked these questions, and you didn’t disclose it.’”

  • ✇Security | CIO
  • Managing AI agents and identity in a heightened risk environment
    Geopolitical tensions are rising. Cyber threats are accelerating. And AI is rapidly expanding the enterprise attack surface. For CIOs and CISOs, the reality is clear: cybersecurity is no longer a defensive function alone. It is now a core element of enterprise resilience. The question leaders should be asking is not simply whether their systems can prevent attacks, but whether their organizations are prepared to detect, contain and recover when something inevitably goes
     

Managing AI agents and identity in a heightened risk environment

19 de Abril de 2026, 22:00

Geopolitical tensions are rising. Cyber threats are accelerating. And AI is rapidly expanding the enterprise attack surface.

For CIOs and CISOs, the reality is clear: cybersecurity is no longer a defensive function alone. It is now a core element of enterprise resilience. The question leaders should be asking is not simply whether their systems can prevent attacks, but whether their organizations are prepared to detect, contain and recover when something inevitably goes wrong.

Ransomware attacks, identity compromise and AI-enabled threats are becoming more sophisticated and more frequent. In this environment, the enterprises that succeed will be those that rethink how security operates from the ground up.

From prevention to resilience

For years, enterprise security strategies focused on prevention. The goal was simple: keep attackers outside the perimeter.

But that model no longer reflects today’s reality.

Modern security strategies increasingly assume that adversaries may already be inside the network, including sophisticated external threat actors that can circumvent even the best perimeter defenses, as well as insider threats. This shift – from perimeter defense to continuous detection and response – is changing how security teams approach everything from infrastructure monitoring to AI deployments.

AI agents, in particular, introduce new layers of complexity, becoming a new category of insider threat. While these systems can automate workflows and unlock significant productivity gains, they can also introduce new vulnerabilities if not carefully governed.

We’ve already seen examples of AI agents behaving unpredictably or making flawed decisions in real-world deployments. Even when systems function as designed, they can create new operational and regulatory risks if guardrails are not in place.  For example, AI agents have deleted entire codebases, approved buggy code, lied to customers and generated unexpectedly large cloud computing bills.

For enterprise leaders, the takeaway is straightforward: AI governance must be a core security discipline. Poorly managed deployments can lead to reputational damage, regulatory exposure, financial loss and operational disruption.

In addition to these internal AI risks, external AI-driven threats are increasing dramatically.  Realistic deepfakes, automated phishing campaigns and advanced ransomware have shown that traditional prevention strategies are no longer sufficient.

The good news is that new tools are emerging to help address these risks. AI-native detection and remediation combined with digital forensics and incident response platforms are enabling organizations to detect and respond to threats faster. These platforms analyze massive volumes of telemetry and behavioral data, helping security teams identify anomalies before they escalate into full-scale incidents.

Identity is the new perimeter

If there is one area where the attack surface has expanded dramatically, it is identity.

As organizations adopt cloud infrastructure, SaaS applications and distributed work environments, identity has become the primary gateway to enterprise systems. Attackers know this, and they increasingly target identity systems as the most efficient path into corporate networks.

That is why Zero Trust identity architectures are becoming essential. Zero Trust assumes that no user, device or system should be automatically trusted. Every request must be verified continuously and access granted based on context, behavior and risk signals.

One piece of this solution is Multi-Factor Authentication (MFA), which should be standard across the enterprise. In addition, modern security platforms increasingly analyze behavioral data to verify human users and identify abnormal activity.  Signals such as keystroke rhythm, geolocation data, time-of-day data and device motion can greatly improve identity accuracy.

Equally important is strong privileged access management (PAM). Elevated privileges should be granted only when necessary and revoked immediately after use, shrinking the vulnerability surface area to the minimum required at any time.  This is even more critical today as AI agents have identities and privileges that are unlikely to be required 24/7.

An emerging trend is correlating data across the various security and posture management silos, including identity (ISMP), cloud (CSPM), application (ASPM) and data (DSPM).  With this, organizations can build unified risk profiles that provide a clearer view of risk and incident progression.  This approach allows security teams to map the full pathway of a potential breach from compromised assets to affected applications, users and exposed data. If a vulnerability appears in an engineering environment, for example, security teams can quickly trace how that exposure could cascade through infrastructure, applications and user accounts.  If a user (or AI agent) is compromised, the relevant at-risk data, applications and cloud environments can be identified.

That level of visibility is becoming essential as enterprise environments grow more complex.

APIs: The backbone of AI — and a major risk

As organizations accelerate AI adoption, APIs are becoming a critical layer of enterprise infrastructure, including the use of Model Context Protocol (MCP) as an orchestration layer. AI systems rely heavily on MCP and various APIs to interact with applications, services and data sources. That means APIs are now one of the most important and most vulnerable components of the enterprise security stack.

A recent API Threatstats report showed that more than 35% of AI vulnerabilities involve APIs. When APIs are poorly secured, they can expose sensitive data, internal logic and authentication mechanisms.

For CIOs leading AI initiatives, this makes API and MCP security a foundational requirement. Organizations must ensure that APIs are continuously monitored, authenticated and protected against misuse.

In many cases, the success or failure of an AI deployment will hinge on how well its API infrastructure is secured.

Preparing for rogue AI agents

Last month, I touched on the rise of autonomous or semi-autonomous AI agents in this column. These systems can perform tasks ranging from software development to customer service to infrastructure management, but their capabilities also introduce new security questions:

How should organizations manage identity for AI agents?
How should their actions be monitored?
And how can enterprises prevent unauthorized or rogue agent activity?

Security strategies must now account for the possibility that AI agents are being manipulated, misconfigured or even intentionally designed to behave maliciously. The rapid adoption of new AI tools is amplifying these concerns. Examples abound in recent months. There are numerous instances in which AI agents, despite their sophisticated algorithms, made poor decisions, exposing significant liabilities for their deployers. 

Platforms such as OpenClaw, one of the fastest-growing AI tools introduced this year, have also spread so quickly that some organizations are restricting their use until stronger safeguards are implemented.

At the same time, smaller companies are gaining access to powerful AI capabilities that were previously available only to large enterprises. That democratization of AI will drive innovation and also increase the potential attack surface across the digital ecosystem.

The CIO imperative

AI adoption is accelerating across every industry. Enterprises are integrating AI agents into development pipelines, business operations and customer engagement systems. But with this opportunity comes responsibility.

For CIOs, the priority is not simply deploying AI technologies; it is deploying them securely.

This means strengthening identity governance, securing APIs, monitoring AI behavior and investing in platforms that provide real-time visibility into enterprise risk. Organizations that navigate this shift successfully will be those that treat cyber resilience as a strategic capability rather than a compliance exercise.

In an era of intelligent systems and autonomous agents, security must go beyond protecting the perimeter; it’s about managing trust across every identity, every API and every system operating inside the enterprise.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

  • ✇Security | CIO
  • 고객지원 챗봇 노린 ‘AI 토큰 무임승차’ 확산…기업 AI 예산 흔든다
    고객 서비스를 위해 AI 에이전트를 도입한 CIO에게 또 하나의 고민이 생겼다. 외부 사용자가 시스템을 교묘히 조작해 기업 비용으로 AI 연산을 수행하도록 만드는 문제다. 이러한 AI 토큰 탈취를 최소화하기 위해 시스템을 잠그는 방법이 없는 것은 아니다. 다만 대부분의 대응책은 단점이 있으며, 자칫하면 해당 시스템의 도입 명분 자체를 약화시킬 가능성도 있다. 이 같은 오남용은 본질적으로 프롬프트 인젝션 공격의 한 형태다. 기업의 AI 비용을 증가시킬 뿐 아니라 투자 대비 수익(ROI)의 가시성을 떨어뜨릴 수 있다. 더 나아가 공격자가 과도한 요청으로 종량제 기반의 고비용 서비스를 과부하 상태로 만들어 수익성을 훼손하는 ‘지갑 서비스 거부(denial of wallet)’ 공격에 기업이 노출될 수 있다. 인포테크 리서치 그룹(Info-Tech Research Group)의 기술 자문 저스틴 세인트모리스는 “이 문제는 빙산의 일각
     

고객지원 챗봇 노린 ‘AI 토큰 무임승차’ 확산…기업 AI 예산 흔든다

16 de Abril de 2026, 20:39

고객 서비스를 위해 AI 에이전트를 도입한 CIO에게 또 하나의 고민이 생겼다. 외부 사용자가 시스템을 교묘히 조작해 기업 비용으로 AI 연산을 수행하도록 만드는 문제다.

이러한 AI 토큰 탈취를 최소화하기 위해 시스템을 잠그는 방법이 없는 것은 아니다. 다만 대부분의 대응책은 단점이 있으며, 자칫하면 해당 시스템의 도입 명분 자체를 약화시킬 가능성도 있다.

이 같은 오남용은 본질적으로 프롬프트 인젝션 공격의 한 형태다. 기업의 AI 비용을 증가시킬 뿐 아니라 투자 대비 수익(ROI)의 가시성을 떨어뜨릴 수 있다. 더 나아가 공격자가 과도한 요청으로 종량제 기반의 고비용 서비스를 과부하 상태로 만들어 수익성을 훼손하는 ‘지갑 서비스 거부(denial of wallet)’ 공격에 기업이 노출될 수 있다.

인포테크 리서치 그룹(Info-Tech Research Group)의 기술 자문 저스틴 세인트모리스는 “이 문제는 빙산의 일각에 불과하다. 훨씬 더 큰 문제를 상징하는 신호일 수 있다”라며 “공격자는 ‘코드를 제공해 준다면, 다른 무엇까지 해줄 수 있는가’라고 생각할 수 있다”고 설명했다.

보안 AI 연합(CoSAI) 회원이자 ACM AI 보안(AISec) 프로그램 위원회 소속인 닉 케일은 비용 구조의 차이를 구체적으로 짚었다. 케일은 “‘내 주문은 어디에 있나? 영업시간은 어떻게 되나?’와 같은 일반적인 고객 응대는 200~300토큰 수준”이라며 “하지만 파이썬으로 연결 리스트를 뒤집어 달라는 요청은 2,000토큰 이상이 쉽게 발생한다. 세션당 비용이 약 10배로 뛰는 셈”이라고 분석했다.

이어 “시스템은 이를 또 하나의 고객 대화로 인식하기 때문에 비용 이상 징후 보고서에 잡히지 않는다”라며 “챗봇 트래픽의 5%만 복잡한 질의를 수행하는 무임승차 이용자라 해도, 분기 실적 검토에서 설명하기 어려운 수준의 예산 공백이 발생할 수 있다”고 전했다.

판단력의 문제

이 사안의 핵심에는 ‘판단력’이 있다. 문제는 챗봇에 이러한 판단력이 거의 없다는 점이다.

케일은 “인간은 맥락적 판단을 기본적으로 내재하고 있다”라며 “하지만 챗봇에는 ‘당신은 도움이 되는 고객 서비스 에이전트다’라는 식의 시스템 프롬프트가 설정돼 있을 뿐이다. 이는 강제 장치가 아니라 일종의 권고 문구에 가깝다. AI판 벨벳 로프와 같은 존재”라고 설명했다.

이어 “이 도구를 조금만 사용해 본 사람이라면 기본적인 대화 구조만으로도 시스템 프롬프트를 우회할 수 있다는 사실을 안다. 현재 기업에서 벌어지는 일이 바로 그것”이라며 “시스템은 세션을 인증할 뿐, 사용자의 의도는 검증하지 않는다”고 지적했다.

그레이하운드 리서치(Greyhound Research)의 수석 애널리스트 산치트 비르 고기아는 이 문제가 앞으로 더 확대될 것으로 내다봤다. 근본적인 책임은 기업에 있다고 진단했다.

고기아는 “기업이 목격하는 것은 챗봇 오남용이 아니라, 고객 서비스라는 이름으로 범용 추론 시스템을 배치한 데 따른 의도치 않은 결과”라며 “이 시스템은 대화형 인터페이스로 설계됐지만, 경제적으로는 개방형 연산 표면처럼 작동한다. 목적과 설계의 불일치가 문제의 출발점”이라고 분석했다.

또한 “모델이 발전한다고 해서 문제가 사라지지는 않을 것이다. 오히려 심화될 가능성이 크다”라며 “AI가 더 강력해지고, 더 쉽게 접근 가능해지며, 더 깊이 내재화될수록 의도된 사용과 의도되지 않은 사용의 경계는 계속 흐려질 것”이라고 전망했다. 이어 “수동적 통제에 의존하는 기업은 비용이 점진적으로 상승하는 현상을 겪게 될 것”이라며 “아키텍처에 능동적 거버넌스를 내장한 기업만이 통제력을 유지할 수 있다. 생성형 AI는 실험 단계에서 운영 단계로 이동하고 있으며, 운영 환경에서는 역량보다 규율이 더 중요하다”고 밝혔다.

포머고브(FormerGov) 전무이사이자 사이버보안 컨설턴트인 브라이언 레빈은 탈옥(jailbreaking)을 리스크 관리의 핵심 과제로 격상해야 한다고 조언했다.

레빈은 “오남용을 예외적 사례로 보지 말고 1차 리스크로 다뤄야 한다”라며 “트래픽의 5%가 의도적이든 아니든 봇 탈옥을 시도하는 상황을 전제로 설계해야 한다”고 말했다. 이어 “이에 선제적으로 대응하는 기업은 AI 예산을 예측 가능하게 유지하고 고객 경험도 보호할 수 있다”라며 “반대로 그렇지 못한 기업은 설명하기 어려운 비용 초과 문제를 해명해야 하는 상황에 놓일 수 있다”고 덧붙였다.

실제 현장에서 벌어지는 AI 토큰 탈취

그렇다면 이러한 챗봇 오남용은 실제로 어떤 모습일까. 소셜미디어에는 이 같은 공격 사례로 추정되는 게시물이 잇따라 올라오고 있다. 링크드인, 레딧, 인스타그램, 엑스(X) 등에서는 아마존 챗봇 오남용 사례가 특히 큰 주목을 받았다. CIO.com은 해당 사례를 직접 재현하기도 했다. 한편 칩otle 사례도 확산됐지만, 칩otle은 해당 게시물이 조작된 것이라고 주장했다.

AI chatbot token freeloading on Amazon's Rufus AI

CIO.com / Foundry

아마존 사례에서는 사이트 방문자가 고객 서비스 봇에 코딩 작업을 요청하는 방식이 활용됐다. 예를 들어 “n번째까지 피보나치 수열을 출력하라”는 요구를 하거나, 스파게티 볼로네제 조리법 전체를 생성하도록 유도하는 식이다.

치폴레 챗봇에서 나왔다고 알려진 사례는 사실 여부가 확인되지 않았다. 해당 게시물의 최초 작성자로 추정되는 인물에게 보낸 메시지에는 답변이 없었고, 치폴레 역시 인터뷰 요청을 거절했다. 치폴레의 외부 커뮤니케이션 매니저 샐리 에번스는 이메일을 통해 “해당 게시물은 포토샵으로 조작된 이미지이며, 챗봇 ‘페퍼(Pepper)’는 생성형 AI를 사용하지도 않고 코딩 기능도 없다”라고 밝혔다. 다만 페퍼가 어떤 기술을 사용하는지, 그리고 왜 해당 이미지가 가짜라고 판단했는지에 대한 추가 질의에는 응답하지 않았다.

실제로 얼마나 심각한 문제인가

이 사안을 기업 CIO의 중대 과제로 봐야 하는지에 대해서는 의견이 엇갈린다. 인포테크 리서치 그룹의 저스틴 세인트모리스는 챗봇이 이처럼 복잡한 질의를 대량으로 처리하게 될 가능성에 회의적인 입장을 보였다.

세인트모리스는 “무료 계정으로 챗GPT를 사용할 수 있는데, 굳이 기업 챗봇을 이용하겠는가”라며 “기업 챗봇은 이런 용도로는 오히려 가장 비효율적인 도구일 수 있다”고 평가했다.

반면 닉 케일은 무료 생성형 AI 챗봇에는 분명한 한계와 제약이 있다고 반박했다. 케일은 “복잡한 질의를 시도하면 매우 빠르게 한계에 부딪힌다”라며 “기업 고객 서비스 챗봇에는 별도의 속도 제한이 없고, 게이트도 없다. 더 강력한 모델을 실행하는 경우가 많다. 사실상 통제되지 않은, 과금 제한도 없는 추론 엔드포인트와 같다”고 지적했다.

다만 케일은 이러한 상황이 CIO.com에게 완전히 새로운 문제는 아니라고 봤다.

케일은 “우리는 이미 같은 장면을 본 적이 있다. 2010년대 초반 REST API 도입 과정에서 기업이 겪었던 사이클과 동일하다”라며 “기업은 엔드포인트를 공개하고 선의의 사용을 가정했다가 남용을 겪은 뒤, 피해가 발생한 이후에야 속도 제한과 API 키 관리를 도입했다”고 설명했다. 이어 “지금은 같은 패턴이 AI 엔드포인트에서 재현되고 있다. 차이점은 요청당 비용이 몇 단계 더 크다는 점이다. REST API를 남용해도 호출당 비용은 극히 적지만, 챗봇에서 복잡한 추론 질의를 실행하면 매번 실질적인 비용이 발생한다”고 분석했다.

그레이하운드 리서치의 산치트 비르 고기아는 남용 비율이 낮더라도 재무적 영향은 빠르게 누적될 수 있다고 경고했다.

고기아는 “구조적으로 위험한 이유는 소수의 행위가 전체 비용을 과도하게 왜곡할 수 있기 때문”이라며 “챗봇 트래픽의 5~8%만 목적 외 고복잡도 질의라 해도, 전체 추론 비용의 4분의 1 이상을 소모할 수 있다”고 설명했다. 이어 “이는 이상 현상이 아니라 토큰 기반 시스템의 작동 방식상 수학적으로 예측 가능한 결과”라며 “다만 비용 급증처럼 보이지 않고 세션당 비용, 세션 길이, 토큰 사용량이 점진적으로 증가하는 형태로 나타나기 때문에 경보가 울리지 않는 경우가 많다”고 덧붙였다.

고기아는 이를 가시성의 실패라고 진단했다. 고기아는 “대부분 기업은 대화 건수, 총 토큰 수, 총비용 같은 활동 지표를 추적하지만, 의도 수준의 경제성을 추적하는 곳은 드물다”라며 “정상적인 고객 지원에서 발생한 비용과 무관한 연산에서 발생한 비용을 구분하지 못한다. 대시보드는 무엇이 일어났는지는 보여주지만, 그것이 일어나야 했는지는 보여주지 않는다. 결국 재무 검토 단계에서야 차이가 드러난다”고 설명했다.

물론 케일이 제기한 두 가지 우려, 즉 통제 불가능한 비용 증가와 통제되지 않은 엔드포인트 문제의 심각성은 기업의 배포 방식과 AI 공급업체 계약 조건에 따라 달라질 수 있다.

가트너의 부사장 애널리스트 나데르 헤네인은 현재 벤더의 요금제 구조가 이러한 탈옥 시도의 영향을 어느 정도 완화한다고 봤다.

헤네인은 “대부분의 대기업은 무제한에 가까운 요금제를 사용하거나 LLM을 내부에서 직접 운영하고 있다”라며 “이 문제가 기업 재무를 크게 흔들 정도는 아닐 것”이라고 전망했다.

리스크 완화를 위한 선택지

챗봇 오남용 위험을 줄이기 위한 가장 직접적인 방법은 고객이 사업과 직접 관련된 질문만 하도록 가드레일을 설계하는 것이다. 그러나 이 과정에서 정당한 고객 질문까지 차단하지 않도록 균형을 맞추는 일은 쉽지 않다. 또한 LLM은 필요할 때 가드레일을 우회하는 경우도 적지 않다.

또 다른 접근법은 추가 AI를 투입해 1차 AI를 감독하거나, 고객 질문 자체가 아니라 단일 응답에서 사용할 수 있는 토큰 수를 제한하는 데 초점을 맞추는 것이다. 다만 토큰 상한선은 사용자가 프롬프트를 여러 개로 나누는 방식으로 우회할 수 있다. 동시에 복잡하지만 정당한 질의까지 차단해 서비스의 비즈니스 가치를 떨어뜨릴 위험도 있다.

AISec의 닉 케일은 여러 대응 방안을 결합해야 한다고 제안했다.

케일은 “실제로 효과가 입증된 방식은 지원 문의처럼 보이지 않는 세션을 식별하는 행동 분석, 단순 요청량을 넘어 맥락까지 고려하는 속도 제한, 그리고 세션별 토큰 사용량을 모니터링해 200토큰 수준의 ‘내 주문은 어디에 있나?’와 2,000토큰이 소요되는 ‘파이썬 스크립트를 작성해 달라’를 구분하는 것”이라고 설명했다. 이어 “하지만 대부분 기업은 고객 서비스 AI에 대해 ‘정교한 자원 남용’을 위협 시나리오로 상정하지 않았기 때문에 이러한 장치를 도입하지 않았다”라며 “이는 와이파이를 개방해 둔 채 이웃이 해당 대역폭으로 암호화폐 채굴을 하고 있다는 사실을 뒤늦게 알게 되는 것과 같은 상황”이라고 비유했다.

포레스터의 부사장 겸 수석 애널리스트 케이트 레겟은 아예 LLM을 배제하고 특정 영역에 특화된 소형 언어 모델을 사용하는 방안을 권고했다. 예를 들어 소비재 기업이라면 원재료 정보처럼 한정된 범위에 집중하는 모델을 구축하는 방식이다.

레겟은 “프라이빗 클라우드나 온프레미스 환경에 배치해 통제할 수 있다”라며 “가장 비용이 많이 드는 방식이지만, 그만한 가치가 있는지는 각 기업의 ROI와 리스크 모델에 달려 있다”고 밝혔다.

인트린식 시큐리티(Intrinsic Security)의 CEO 게리 롱사인은 제출된 질의를 사전에 검토하는 두 번째 LLM을 두는 방식도 현실적인 대안이 될 수 있다고 봤다.

롱사인은 “추가 토큰 비용과 응답 지연이 발생할 수 있다”라며 “다만 사용자 프롬프트와 병렬로 검토를 수행하고, 자체 호스팅 LLM을 활용하면 일부 완화가 가능하다”고 설명했다.

CIO가 어떤 대응 전략을 선택하든, 보다 근본적인 질문에 대한 답이 필요하다는 지적도 나온다. 고객 서비스 AI 도입의 정확한 비즈니스 목적과 기대 성과가 무엇인지 명확히 해야 한다는 것이다.

무어 인사이트 앤드 스트래티지(Moor Insights and Strategy)의 수석 애널리스트 제이슨 앤더슨은 “기업은 이제 고객 서비스 AI를 단순한 지원 비용이 아니라 새로운 판매 채널로 인식해야 한다”라며 “많은 지원 솔루션이 문의 전환 감소 등 비용 절감 지표 중심으로 평가되고 있다. 앞으로는 수익 지표와 목표 설정도 함께 논의해야 한다”고 말했다.

매시브스케일AI(MassiveScale.AI)의 CEO 조슈아 우드러프는 CIO와 조직이 거버넌스의 기본 작업에 직접 나서야 한다고 강조했다.

우드러프는 “범위 정의, 접근 통제, 사용 사례 경계 설정과 같은 기본 작업이 실제 거버넌스의 모습”이라며 “눈에 띄는 혁신으로 보도되지는 않지만, 보도자료에 실릴 만한 화려한 작업도 아니지만, 고객 서비스 봇과 기업 로고를 단 우발적 무료 AI 서비스 사이를 가르는 결정적 차이”라고 밝혔다.
dl-ciokorea@foundryco.com

  • ✇Security | CIO
  • Why CIOs are moving away from legacy consulting in the AI era
    The structural limits of traditional enterprise consulting are being exposed by artificial intelligence, and the breakdown is occurring at the seams between strategy and execution. As organizations race to adopt AI while managing an increasingly complex cybersecurity situation, the gap between what legacy firms promise and what they can actually deliver has become impossible to ignore. Traditional consulting models were designed for a world where transformation could be
     

Why CIOs are moving away from legacy consulting in the AI era

14 de Abril de 2026, 09:18

The structural limits of traditional enterprise consulting are being exposed by artificial intelligence, and the breakdown is occurring at the seams between strategy and execution. As organizations race to adopt AI while managing an increasingly complex cybersecurity situation, the gap between what legacy firms promise and what they can actually deliver has become impossible to ignore.

Traditional consulting models were designed for a world where transformation could be sequenced. Strategy came first, followed by design, then implementation, and finally, risk management. AI collapses those timelines because enterprises are deploying models in weeks, not years, and the risk surface is evolving in real time. But despite this constant state of change, many large consulting firms still send in strategy teams that define AI ambitions without understanding how these models have been built and deployed, or how they may be attacked. Security is brought in after the fact, with consultants retrofitting controls onto systems that were never designed to be secure. Accountability is fragmented, and no single party owns the outcome end-to-end.

The economics of traditional consulting compounds the problem. Large teams with continuous upsell cycles do not align with what CIOs need right now, which is speed, precision, and accountability. The result is a growing distance between what gets presented in the boardroom and what holds up in production.

AI adoption is already taking place, regardless of whether the organization has a formal governance policy. Business units are experimenting. Engineers are integrating models. Data is moving in ways that are not fully visible to security or leadership. Right now, CIOs’ priority is to get their hands around what is already running across the enterprise and determine whether it is safe. Organizations need to know where AI is already embedded, and what it’s doing. What data is being exposed, and to whom? Who owns the risk when a model behaves unexpectedly? Answering these questions requires technical depth and operational experience that traditional consulting structures are not built to provide.

The same dynamic is playing out on the security side. AI is following the trajectory that DevOps followed a decade ago, when security was bolted on after the fact and the industry had to evolve toward DevSecOps. That model won’t work for AI, because AI is moving much faster, and the consequences of getting it wrong are greater. Security must be embedded into model architecture and selection, data pipelines and governance, access controls, and runtime monitoring from the start. Counterintuitively, this model enables IT to accelerate AI service delivery because it can avoid much of the costly rework and remediation cycles that follow a breach or a compliance failure.

Closing the gap between knowing and doing requires a different kind of engagement model with firms built with people who have actually held the roles on which they’re advising, and who have already deployed, secured, and operationalized secure, compliant AI. The incentive structure is different, too, because in this next generation of firms, the goal is not to create dependency, but to solve the problem with a solution that will not require ongoing consulting support to run.

Arcova, formerly MorganFranklin Cyber, rebranded in 2026 to reflect exactly this shift. As enterprises face the convergence of cybersecurity risk and AI adoption, Arcova operates as an embedded partner that can secure and modernize simultaneously, bringing practitioner-level accountability to engagements rather than the arm’s-length advisory model that legacy firms have long relied on.

The firms that will define the next era of enterprise consulting are not the ones with the largest partner hierarchies or the most standardized playbooks. They are the ones where the people doing the work have owned the problem before. In an environment where AI risk is real, present, and already inside the enterprise, the qualities that define these next-generation firms will not just be differentiators. They have become requirements.

Learn more about Arcova here.

  • ✇Security | CIO
  • AI token freeloaders are coming for your customer support chatbot
    CIOs deploying AI agents for customer service have one more thing to worry about: external users tricking the system into delivering AI computations on your dime.  Although there are ways to lock down these systems to minimize AI token theft, they all have downsides, including the possibility of undermining the business case for these very systems. Essentially a form of prompt injection attack, such misuse can not only increase enterprise AI bills but also make ROI vi
     

AI token freeloaders are coming for your customer support chatbot

9 de Abril de 2026, 06:00

CIOs deploying AI agents for customer service have one more thing to worry about: external users tricking the system into delivering AI computations on your dime. 

Although there are ways to lock down these systems to minimize AI token theft, they all have downsides, including the possibility of undermining the business case for these very systems.

Essentially a form of prompt injection attack, such misuse can not only increase enterprise AI bills but also make ROI visibility murkier. Moreover, it can expose enterprises to “denial of wallet” attacks, in which attackers overload costly pay-as-you-go services with excessive requests to damage the bottom line.

“This is only the tip of the iceberg of your risks. It is a potential symbol of a much bigger problem,” says Justin St-Maurice, a technical counselor at Info-Tech Research Group. A potential attacker might ask, “If they are willing to give me code, what else are they willing to do for me?”

“A normal customer service interaction of ‘Where’s my order? What are your hours?’ runs maybe 200 to 300 tokens. Someone asking the bot to reverse a linked list in Python is generating more than 2,000 tokens easy. That’s roughly a 10x cost multiplier per session,” says Nik Kale, member of the Coalition for Secure AI (CoSAI) and ACM’s AI Security (AISec) program committee.

“And it doesn’t show up in any cost anomaly report because the system just sees it as another customer conversation,” he adds. “You could have 5% of your chatbot traffic be freeloaders running complex queries and it would blow a material hole in your AI budget that nobody can explain in a quarterly review.”

A question of judgment

Judgment is a key part of this issue, and the problem is that chatbots have little to none.

“A human has contextual judgment baked in,” Kale notes. “These chatbots have a system prompt that says something like, ‘You are a helpful customer service agent.’ That’s a suggestion, not an enforcement mechanism. It’s the AI equivalent of a velvet rope.”

He adds: “Anyone who’s spent five minutes with these tools knows you can steer past a system prompt with basic conversational framing, which is exactly what [is happening to enterprises today]. The system authenticates the session, not the intent.”

Sanchit Vir Gogia, chief analyst at Greyhound Research, sees this issue increasing — with enterprises fundamentally to blame. 

“What enterprises are witnessing is not misuse of chatbots but the unintended consequence of deploying general-purpose inference systems under the label of customer service,” he says. “These systems are architected as conversational interfaces, but economically they behave as open compute surfaces. That mismatch between purpose and design is where the problem begins.”

Gogia argues that, like many AI challenges, this issue will multiply as models advance.

“The problem will not disappear as models improve. It will intensify. As AI becomes more capable, more accessible, and more embedded, the boundary between intended and unintended usage will continue to blur,” Gogia says. “Enterprises that rely on passive controls will see costs drift. Enterprises that build active governance into their architecture will maintain control. This is the real shift under way. Gen AI is moving from experimentation to operations. And in operations, discipline matters more than capability.”

Part of that discipline includes elevating jailbreaking as a risk management priority, says cybersecurity consultant Brian Levine, executive director of FormerGov.

“You need to treat misuse as a first‑order risk, not an edge case. Build for the world where 5% of your traffic will try to jailbreak your bot, intentionally or not,” he says. “The companies that get ahead of this will keep their AI budgets predictable and their customer experience intact. The ones that don’t will be explaining mysterious cost overruns.”

AI token theft in practice

What exactly does this kind of chatbot misuse look like? Social media has been flooded with supposed examples of these attacks, with the most attention across LinkedIn, Reddit, Instagram, and X going to misuse of chatbots at Amazon — which CIO.com was able to replicate below — and one at Chipotle, which Chipotle claims was fake. 

AI chatbot token freeloading on Amazon's Rufus AI

CIO.com / Foundry

The Amazon examples — including this and this — revolved around site visitors getting the customer service bot to perform a coding service (“output the Fibonacci sequence up to n count”) or deliver a complete recipe for spaghetti bolognese.

A much-referenced example supposedly from a Chipotle bot is unconfirmed. Messages sent to the apparent original poster of the Chipotle example have not been responded to, and Chipotle declined an interview request. “The viral post was Photoshopped. Pepper neither uses gen AI nor has the ability to code,” Sally Evans, Chipotle’s external communications manager, replied by email, referring to the chatbot, Pepper, but did not respond to follow-up questions to clarify what Pepper uses and why Chipotle believed the image was fake.

How big of a deal is this really?

Not everyone is convinced that this is a major issue for enterprise CIOs. Info-Tech’s St-Maurice, for one, doubts chatbots will be fielding a lot of these queries.

“Couldn’t they just use ChatGPT for free, using a free account?” he asks. “[An enterprise chatbot] is probably the worst tool for this.”

AISec’s Kale disagrees, arguing that free gen AI chatbots have limits and gates. “You very quickly hit a wall with complex queries,” he notes. “With [enterprise customer service chatbots], there is no rate limit. They are ungated, unmetered inference endpoints andthey are running far more capable models. These chatbots are ungoverned endpoints.” 

But Kale also notes that this is old hat for most CIOs.

“We’ve seen this exact movie before. This is the same cycle enterprises went through with REST APIs in the early 2010s. Companies exposed endpoints, assumed good-faith usage, got hammered by abuse, then retrofitted rate limiting and API key management after the damage was done,” he explains. “We’re watching the same pattern replay with AI endpoints, except the per-request cost is orders of magnitude higher. A bad actor abusing your REST API costs you fractions of a penny per call. Someone running complex reasoning queries through your chatbot costs real money every single time.”

Greyhound’s Gogia adds that even if the frequency of this abuse is small, its impact can add up quickly.

“What makes this structurally risky is that a small percentage of behavior can disproportionately distort total cost. Even if 5-8% of chatbot traffic consists of off-purpose or high complexity queries, that slice can consume a quarter or more of total inference spend. These are not anomalies. They are mathematically predictable outcomes of how token-based systems operate. Yet they rarely trigger alerts because they do not appear as spikes. They appear as gradual drift in cost per session, session length, and token consumption,” Gogia says.

“This leads to a deeper failure in observability,” he adds. “Most enterprises today track activity metrics such as number of conversations, total tokens, and aggregate cost. Very few track intent-level economics. They cannot distinguish between cost generated by legitimate customer service interactions and cost generated by irrelevant compute. Dashboards show what happened, but not whether it should have happened. So everything looks normal until financial reviews expose the gap.”

For many CIOs, the degree to which Kale’s two concerns — out-of-control costs and bots as ungoverned endpoints — are true depends on both their specific deployments and AI supplier contracts. 

Here, Gartner VP analyst Nader Henein sees current vendor pricing tiers softening the impact of such jailbreaking efforts. 

“Most large organizations either have an all you can eat plan or run their LLMs internally, so I doubt this is going to break the bank,” he says.

Mitigating the risk

The most straightforward approach to mitigate the risk of chatbot misuse is to craft guardrails that restrict customers to questions directly related to the business. But such limits are challenging to construct without unintentionally blocking legitimate customer questions. Moreover, LLMs often sidestep guardrails when they are most needed

Another approach could involve enlisting additional AI to oversee front-line AI, or to focus not on customer queries but on limiting the number of tokens that can be used for any single answer. Token limits, however, could still be circumvented by abusers by breaking prompts into smaller parts. Complex legitimate queries could also be inadvertently prohibited by putting a ceiling on token use, limiting the business value of the service.

AISec’s Kale recommends a combination of tactics. 

“The patterns that actually work are behavioral analysis to flag sessions that don’t look like support queries, contextual rate limiting that goes beyond just volume, and token-level usage monitoring per session that can distinguish a 200-token ‘Where’s my order?’ from a 2,000-token ‘Write me a Python script,’” he says. “But most companies haven’t implemented any of this because they never threat-modeled ‘sophisticated resource abuse’ for their customer service AI. It’s the AI equivalent of leaving your Wi-Fi open and discovering your neighbor’s been running a cryptomining operation on your bandwidth.”

Kate Leggett, VP and principal analyst at Forrester, advises dumping LLMs entirely and using small language models focused on specific segments, such as ingredients at a consumer packaged goods company.

“You can host it on a private cloud or even on-prem and you can lock it down,” she says. “That is the most expensive way to do it. Is it worth it? That comes down to your ROI and risk model.”

Gary Longsine, CEO of Intrinsic Security, believes enlisting a second LLM to review submitted queries could be reasonably effective. “But it would introduce a token cost and possibly a response time delay,” he says. “Those could be mitigated somewhat by running the review in parallel with the user prompt, and by using a self-hosted LLM to do the review.”

However CIOs choose to deal with this issue, the larger implications must be addressed — namely, what exactly is the business purpose, and expected outcomes, of your customer service AI implementation?

“Companies need to recognize that this is now a new selling channel for them, not just a customer support cost,” says Jason Andersen, principal analyst at Moor Insights and Strategy. “A lot of these support solutions are primarily measured on cost reductions, such as deflection. Will they now have revenue measures and quotas?”

In the meantime, CIOs and their teams need to roll up their sleeves and do the grunt work of AI governance, says Joshua Woodruff, CEO of MassiveScale.AI.

“The boring work — scope definition, access controls, use case boundaries — is what governance actually looks like in practice,” he says. “It’s not glamorous work and it’s not making headlines for being innovative. It doesn’t make the press release. But it’s the absolute difference between a customer service bot and an accidental free AI service with a corporate logo on it.”

  • ✇Security | CIO
  • War is forcing banks toward continuous scenario planning
    War is already changing the operating conditions for banks faster than most planning systems can respond. This article uses banking as its primary lens, but the underlying challenge — planning systems that cannot absorb change fast enough — applies across most industries. That is the real issue. I have spent a large part of my career working on planning systems, portfolio decisions, and executive teams, trying to make sense of change while the ground was already movi
     

War is forcing banks toward continuous scenario planning

8 de Abril de 2026, 09:14

War is already changing the operating conditions for banks faster than most planning systems can respond. This article uses banking as its primary lens, but the underlying challenge — planning systems that cannot absorb change fast enough — applies across most industries.

That is the real issue.

I have spent a large part of my career working on planning systems, portfolio decisions, and executive teams, trying to make sense of change while the ground was already moving beneath them. One lesson keeps repeating itself: organizations rarely fail because they did not produce a plan. They fail because reality changed faster than the planning system could absorb.

That is where banks find themselves today.

As of April 2026, war is no longer a geopolitical backdrop to financial services. It is a live, operating variable that shapes markets, liquidity, sanctions exposure, and cross-border payments in real time. In a Reuters interview, IMF Managing Director Kristalina Georgieva said the war will mean “higher prices and slower growth” even if it ends soon, because the disruption to energy and supply channels is already feeding through the system. The same report noted that the disruption around the Strait of Hormuz is affecting a route that carries roughly one-fifth of global oil and gas flows.

For banks, that is not the macro context.

It is the operating environment.

Why static planning is now dangerous

Our current environment is why continuous scenario planning has moved from a good discipline to a core requirement. In wartime, annual planning is dangerous. Quarterly planning is slow. I would argue that anything less than weekly planning is now dangerous for globally exposed banks, because the variables that matter most can change inside a single board cycle.

The existing planning model was built for a different world. A bank could set an annual strategy, align budgets, review quarterly, run periodic stress tests, and treat disruptions as exceptions. But regulators and markets are now signaling that the exception has become the norm. The European Central Bank has made resilience to geopolitical risks and macro-financial uncertainty a supervisory priority. In 2026, it will conduct a reverse stress test on geopolitical risks across 110 directly supervised banks. That matters because the ECB is not simply handing banks a standard scenario and asking whether they survive. It is asking each institution to define the path by which geopolitical events could drive severe capital depletion.

That is a profound shift.

It means supervisors are effectively saying: do not just tell us you can survive a known shock. Show us that you can think dynamically about your own vulnerabilities before the next shock hits.

How shocks actually hit a bank

The problem with war-driven disruption is that it does not fall neatly into one category. It arrives as a chain reaction.

A conflict intensifies. Energy prices rise. Inflation expectations shift. Rate cuts get delayed. Funding assumptions change. Corporate borrowers in the transport, manufacturing, agriculture, and energy-intensive sectors are under greater pressure. Hedging behavior changes. Clients move money differently. Sanctions expand. Cross-border payment flows become more complex. Compliance teams must review exposures and counterparty relationships faster. Treasury, risk, operations, technology, and the business all need answers at once.

That is not a single issue.

It is a system problem.

And that is exactly where static planning breaks. The arithmetic is moving while the plan is standing still.

Take one realistic example. Reuters reported that UBS lowered its 2026 S&P 500 target because the Middle East conflict pushed oil prices higher, increased economic uncertainty, and delayed expected Federal Reserve cuts. Since the outbreak of the Iran war on February 28, the S&P 500 has fallen about 3.9 percent, according to that report.

Now translate that into a bank’s actual management problem.

If oil stays elevated for months, that is not simply a market call. It affects the credit quality outlook for specific sectors. It changes treasury assumptions. It alters customer behavior. It can simultaneously pressure deposit flows, lending appetite, capital allocation, and operating costs. That broader financial stability concern was echoed by ECB Governing Council member Fabio Panetta, who warned Reuters that the energy shock was raising financial stability risks. If the institution is also in the middle of a cost program, a core modernization effort, or a remediation exercise, the same scarce people may be required in multiple places simultaneously.

This is the kind of moment when the board asks a perfectly reasonable question:

What do we stop, what do we protect, what do we accelerate, and what is the consequence of each choice?

Traditional planning does not answer that question well. It usually answers a different question: what did we think the year would look like when we approved the budget?

What continuous scenario planning changes

Continuous scenario planning starts somewhere else. It asks what changed, which constraints moved, which dependencies are now binding, and what tradeoffs leadership has to make before the cost compounds.

Annual planning is calendar-based.

Continuous scenario planning is trigger-based.

Annual planning assumes the world will pause for review.

Continuous scenario planning assumes the world will not.

That matters because several of the pressures hitting banks now are not transient annoyances. They are structural conditions. The IMF is warning about slower growth and higher inflation from the current conflict. Andrew Bailey, speaking as chair of the Financial Stability Board, warned that “frictions in international payments have the potential to act as a driver of fragmentation of the global system” in a March FSB speech.

Put differently, banks are not dealing with one shock. They are dealing with a more fragmented operating order, a point the ECB and ESRB underscored when they warned that geopolitical shocks can lead banks and non-banks to reduce lending, especially cross-border exposures.

Sanctions and payments are now operational risk

That fragmentation shows up in sanctions and payments as clearly as anywhere and is no longer just a back-office compliance matter. When sanctions expand or diverge across jurisdictions, global banks have to think through counterparty exposure, corridor reliability, client obligations, correspondent relationships, liquidity flows, and operational bottlenecks together.

That is not theoretical. In February 2026, Reuters reported that U.S. regulators proposed cutting Swiss private bank MBaer off from the U.S. financial system over alleged links tied to Iran, Russia, and Venezuela, a reminder that sanctions risk can quickly become an existential operating issue for a bank. Bailey’s warning about payment friction matters because it points to something many leadership teams still underestimate: the plumbing itself is becoming geopolitical.

Imagine the practical version. A multinational bank with operations in Europe, Asia, and the Gulf wakes up to a sanctions escalation tied to a military event. Certain counterparties become restricted. Some clients need urgent payment rerouting. Compliance wants more review. Relationship managers want client continuity. Treasury wants to understand liquidity effects. Technology teams are suddenly dealing with workflow changes and control logic. The executive committee wants to know, by tomorrow morning, the revenue and risk impacts, and which commitments elsewhere in the portfolio need to be moved now.

That is not a better dashboard problem.

That is a scenario problem.

The CIO mandate is changing

Many banks still understate the CIO’s role here. This is not only a strategy issue or a risk issue. It is a systems issue. The current architecture in many institutions was not built to support live, cross-functional consequence modeling. Data sits in separate systems. Capacity is tracked in one place, financial assumptions in another, execution commitments in yet another, and regulatory obligations in yet another. During stable periods, the seams are tolerable. Under geopolitical pressure, they become expensive.

The CIO’s mandate is changing as a result. It is no longer enough to ensure that the bank has strong systems of record. The harder challenge is to make those systems usable when decisions are being made.

That means three things:

  1. Reduce the distance between risk, finance, payments, and portfolio data. If each function has to build its own answer after every shock, the bank is already late.
  2. Create a planning environment where leadership can test scenarios across the same set of constraints. Not treasury in one model, compliance in another, and transformation in a slide deck. One environment. Shared assumptions. Visible tradeoffs.
  3. Shift from reporting cycles to trigger cycles. If sanctions change, an energy shock, or a payments disruption can alter the bank’s risk posture in days, then the planning system has to move on that clock as well.

That is the directional answer for CIOs. Build the connective tissue that lets the institution compare consequences before separate functions lock in separate responses.

The good news is that most banks already have the raw ingredients. ERP systems hold the financial data. PPM or PMO platforms track the portfolio of change. SPM tools carry strategic priorities. Project plans, capacity models, and even well-structured spreadsheets contain the operational detail. The data problem in most institutions is not one of absence. It is one of fragmentation. What has changed is that the technology to connect those sources into a common planning environment now exists, and AI-driven interfaces mean that a CFO, CRO, or CEO can query that connected environment in plain language, asking the kind of questions that used to require a team of analysts and a week of lead time. The barrier is no longer technical. It is organizational will.

How banks actually build the capability

How do banks get there without turning this into another expensive transformation initiative?

In my experience, they do not start by modeling everything.

They start by choosing one decision arena where geopolitical shocks already create visible stress. It might be sanctions and payments. It might be energy-sensitive credit exposure. It might be liquidity and funding assumptions. The point is to begin where a shock can simultaneously move earnings, risk, and operations.

From there, the path is more practical than many leaders assume:

  1. Connect the few data streams that determine the decision. Risk, finance, payments, capacity, and the active change portfolio usually matter more than another round of presentation material.
  2. Define the triggers that force a scenario review. A sanctions change. A corridor disruption. A commodity spike. A rate shift. A supervisory action. If the trigger is real, the review should be immediate, not deferred to the next reporting cycle.
  3. Establish a weekly executive cadence for scenario comparison. Not a status meeting. A decision meeting. The question is not what changed in each silo. The question is: what does the bank now need to protect, slow, accelerate, or stop?
  4. Make tradeoffs explicit. If the institution prioritizes liquidity resilience, what gets delayed? If it protects payment continuity, which transformation resources move? If it accelerates one control program, what capacity disappears elsewhere?

Continuous scenario planning now becomes operational, not as a new planning religion, but as a repeatable discipline for making better decisions under pressure.

Banks do not need to perfect this across the entire enterprise on day one. They need to become faster and more coherent in the places where shocks are already forcing hard choices. That is usually how the capability starts.

The market is already moving

The question is not whether the bank has data. Of course it does. The question is whether it can connect the data quickly enough to calculate the next consequence before management is forced into guesswork.

The market examples from the last year should be sobering. Reuters reported that tariff turmoil cast a pall over European banks’ earnings outlook in 2025 as trade stress hit confidence, credit expectations, and sector valuations. And in March 2026, J.P. Morgan warned via Reuters that HSBC and Standard Chartered were among the European banks most exposed to the Middle East conflict, underscoring how unevenly geopolitical shocks can land across institutions and portfolios.

The market does not wait for the next planning cycle.

The stronger signal is operational, not rhetorical. Supervisors are forcing banks to design reverse-geopolitical stress scenarios because standardized exercises are no longer sufficient. Markets are repricing institutions unevenly when conflict exposure becomes clearer. And when trade shocks or sanctions changes hit, the effect does not stay inside one function. It considers earnings expectations, sector risk, payments, and capital allocation simultaneously.

What real-time scenario planning looks like in practice

What does that look like in practice?

It means a bank can stop arguing over whether a shock matters and start testing what it does.

Imagine a leadership team on a Tuesday morning after an overnight escalation. Oil is up. Sanctions lists are changing. A key payments corridor is slowing. The traditional response is a flurry of calls, separate spreadsheets, and each function building its own partial answer. Treasury models liquidity. Risk reviews sector exposure. Compliance checks counterparties. Technology assesses workflow impact. The executive committee gets fragments, not a usable picture.

Instead of asking each team for a static update, leadership can test a live set of scenarios. What happens if oil stays elevated for ninety days instead of thirty? Which client segments move from watchlist to immediate concern? What if sanctions are widened to include adjacent counterparties, or if a payment route becomes unreliable? Which programs lose capacity first because the same people are now needed for control changes, client support, and remediation work? Which commitments should be protected because they stabilize the bank, and which should be deferred because they consume scarce talent without improving resilience?

That is the difference. The point is not simply to produce a more elegant plan. The point is to give leadership a way to compare consequences before they commit.

A good continuous scenario discipline also forces hard choices into the open. If the bank protects liquidity and sanctions readiness, what gets delayed? If it accelerates one modernization program by reducing operational fragility, what other initiative costs people’s lives? If a region becomes harder to serve profitably under new risk assumptions, what does that mean for revenue plans, client coverage, and capital allocation? Those are painful questions, but they are far less painful when asked early.

This is why real-time scenario planning matters in wartime conditions. It helps banks move from narrative to arithmetic. From reaction to comparison. From isolated function views to system-level tradeoffs. And from vague resilience language to specific choices made today, this week, not next quarter. That urgency is consistent with what Reuters described as conflict-driven market repricing, elevated oil, and delayed expected Fed cuts.

It matters because the bank that can compare plausible paths quickly has a real advantage over the bank that can only defend last quarter’s assumptions more eloquently.

What the board should conclude now

In practical terms, continuous scenario planning lets leadership test questions that are now central to resilience:

  • If energy stays elevated for two more quarters, which sectors deserve immediate credit scrutiny?
  • If rates remain higher for longer, which transformation initiatives still deserve capital, and which should be slowed?
  • If sanctions widen, which payment corridors become operationally fragile?
  • If the same teams are needed for compliance changes, cost reduction, and modernization, where will overload appear first?
  • And perhaps most importantly, what does management choose to protect when it cannot protect everything?

That is the heart of the issue. In times of war, the planning challenge is not prediction. It is prioritization under pressure.

If I were making the board-level case for urgency, I would put it this way: banks do not need another quarter of elegant commentary about uncertainty. They need a weekly discipline that shows which assumptions have broken, which exposures have changed, which commitments no longer hold, and which decisions cannot wait. Without that, the institution is not planning. It is narrating.

The institutions that handle this best will not be the ones that guessed the future most precisely. They will be the ones who built a planning discipline capable of absorbing change without freezing. They will recognize that resilience now depends on decision speed, not just capital strength. And they will understand that planning cannot remain an annual ritual in a world where geopolitical shocks are continuous.

War broke that model.

Banks now need one that moves.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

  • ✇Security Boulevard
  • Proven incident response and business continuity strategy Shweta Dhole
    From cybersecurity breaches to natural disasters, disruptive events can occur suddenly and without warning. As a result, it is crucial for organizations to develop resilient plans that not only respond to incidents in real time but also ensure long-term operational survivability. This article examines the concepts of incident response and business continuity, exploring their differences […] The post Proven incident response and business continuity strategy first appeared on TrustCloud. The post
     

Proven incident response and business continuity strategy

6 de Abril de 2026, 04:09

From cybersecurity breaches to natural disasters, disruptive events can occur suddenly and without warning. As a result, it is crucial for organizations to develop resilient plans that not only respond to incidents in real time but also ensure long-term operational survivability. This article examines the concepts of incident response and business continuity, exploring their differences […]

The post Proven incident response and business continuity strategy first appeared on TrustCloud.

The post Proven incident response and business continuity strategy appeared first on Security Boulevard.

❌
❌