What is data analytics?

Data analytics focuses on gleaning insights from data. It comprises the processes, tools, and techniques of data analysis and management, and its chief aim is to apply statistical analysis and technologies on data to find trends and solve problems. Data analytics has become increasingly important in the enterprise to shape business processes and improve decision-making and business results.

Data analytics draws from a range of disciplines, including computer programming, mathematics, and statistics, to perform analysis on data in an effort to describe, predict, and improve performance. So to ensure robust analysis, data analytics teams leverage a range of data management techniques, including data mining, data cleansing, data transformation, data modeling, and more.

What is AI data analytics?

AI data analytics is a rapidly growing specialty within data analytics that applies AI to support, automate, and simplify data analysis. It leverages ML, natural language processing, and data mining, along with foundational models and chat assistance for predictive analytics, sentiment analysis, and AI-enhanced business intelligence. AI tools can be used for data collection and data preparation, while ML models can be trained to extract insights and patterns.

The four types of data analytics

Analytics breaks down broadly into four types: descriptive analytics attempts to describe what has transpired at a particular time; diagnostic analytics assesses why something has happened; predictive analytics ascertains the likelihood of something happening in the future; and prescriptive analytics provides recommended actions to take to achieve a desired outcome.

To explore these more specifically, descriptive analytics uses historical and current data from multiple sources to describe the present state, or a specified historical state, by identifying trends and patterns. Business analytics is the purview of business intelligence (BI). Diagnostic analytics uses data, often generated via descriptive analytics, to discover the factors or reasons for past performance. Predictive analytics applies techniques such as statistical modeling, forecasting, and ML to the output of descriptive and diagnostic analytics to make predictions about future outcomes. Predictive analytics is often considered a type of advanced analytics, and frequently depends on ML and/or deep learning. And prescriptive analytics is another type of advanced analytics that involves the application of testing and other techniques to recommend specific solutions that will deliver outcomes. In business, predictive analytics uses ML, business rules, and algorithms.

Data analytics methods and techniques

Data analysts use a number of methods and techniques to analyze data. According to Emily Stevens, managing editor at CareerFoundry, seven of the most popular include:

1. Regression analysis: A set of statistical processes used to estimate the relationships between variables to determine how changes to one or more might affect another, like how social media spending might affect sales.
2. Monte Carlo simulation: A mathematical technique, frequently used for risk analysis, that relies on repeated random sampling to determine the probability of various outcomes of an event that can’t otherwise be readily predicted due to degrees of uncertainty in its inputs.
3. Factor analysis: A statistical method for taking a massive data set and reducing it to a smaller, more manageable one to uncover hidden patterns, like when analyzing customer loyalty.
4. Cohort analysis: A form of analysis in which a dataset is broken into groups that share common characteristics, or cohorts, for analysis like understanding customer segments.
5. Cluster analysis: A statistical method in which items are classified and organized into clusters in an effort to reveal structures in data. Insurance firms might use cluster analysis to investigate why certain locations are associated with particular insurance claims, for instance.
6. Time series analysis: A statistical technique in which data in set time periods or intervals is analyzed to identify trends over time, such as weekly sales numbers or quarterly sales forecasting.
7. Sentiment analysis: A technique that uses natural language processing, text analysis, computational linguistics, and other tools to understand sentiments expressed in data, such as how customers feel about a brand or product based on responses in customer forums. While the previous six methods seek to analyze quantitative or measurable data, sentiment analysis seeks to interpret and classify qualitative data by organizing it all into themes.

Data analytics tools

Data analysts use a range of tools to aid them surface insights from data. Some of the most popular include: 

• Apache Spark: An open source data science platform to process big data and create cluster computing engines. 
• AskEnola AI: A conversational analytics tool for business users.
• Data analysis with ChatGPT: OpenAI’s chatbot can generate code to perform data analysis, transformation, and visualization tasks using Python.
• dbt: An open source analytics engineering tool for data analysts and engineers.
• Domo Analytics: A BI SaaS platform to gather and transform data.  
• Excel: Microsoft’s spreadsheet software for mathematical analysis and tabular reporting. 
• Julius AI: An AI assistant to analyze spreadsheets and databases.
• Knime: A free and open source data cleaning and analysis tool for data mining.
• Looker: Google’s data analytics and BI platform. 
• MySQL: An open source relational database management system to store application data used in data mining.
• Observable: A data analysis platform with AI tools for exploratory data analysis and data visualization.
• Orange: A data mining tool ideal for smaller projects.
• Power BI: Microsoft’s data visualization and analysis tool to create and distribute reports and dashboards. 
• Python: An open source programming language popular among data scientists to extract, summarize, and visualize data. 
• Qlik: A suite of tools to explore data and create data visualizations. 
• R: An open source data analytics tool for statistical analysis and graphical modeling. 
• RapidMiner: A data science platform that includes a visual workflow designer. 
• SAS: An analytics platform for business intelligence and data mining. 
• Sisense: A popular self-service BI platform. 
• Tableau: Data analysis software from Salesforce to create data dashboards and visualizations.

Data analytics vs. data science

Data analytics is a component of data science used to understand what an organization’s data looks like. Generally, the output of data analytics are reports and visualizations. Data science takes the output of analytics to study and solve problems. The difference between data analytics and data science is often about timescale. Data analytics describes the current or historical state of reality, whereas data science uses that data to predict and/or understand the future.

Data analytics vs. data analysis

While the terms data analytics and data analysis are frequently used interchangeably, data analysis is a subset of data analytics concerned with examining, cleansing, transforming, and modeling data to derive conclusions. Data analytics includes the tools and techniques used to perform data analysis.

Data analytics vs. business analytics

Business analytics is another subset of data analytics. It uses data analytics techniques, including data mining, statistical analysis, and predictive modeling, to drive better business decisions. Gartner defines business analytics as solutions used to build analysis models and simulations to create scenarios, understand realities, and predict future states.

Data analytics examples

Organizations across all industries leverage data analytics to improve operations, increase revenue, and facilitate digital transformations. Here are three examples:

UPS transforms air cargo operations with data, AI: UPS’s Gateway Technology Automation Platform (GTAP) uses AI and digital asset tracking to reduce costs, improve on-time performance, and enhance operational safety at its Worldport air hub.

NFL leverages AI and predictive analytics to reduce injuries: The NFL’s Digital Athlete platform leverages AI and ML to run millions of simulations of in-game scenarios, using video and player tracking data to identify the highest risk of injury during plays, and develop individualized injury prevention courses.

Fresenius Medical Care anticipates complications with predictive analytics: Fresenius Medical Care, which specializes in providing kidney dialysis services, is pioneering the use of a combination of near real-time IoT data and clinical data to predict when kidney dialysis patients might suffer a potentially life-threatening complication called intradialytic hypotension (IDH).

Data analytics salaries

According to data from PayScale, the average annual salary for a data analyst is $70,384, with a reported range from $51,000 to $95,000. Salary data on similar positions include:

PayScale also identifies cities where data analysts earn salaries that are higher than the national average. These include San Francisco (24.2%), Seattle (10.2%), and New York (9.5%).

기업들은 AI 실험 단계를 빠르게 넘어 실제 운영으로 전환하고 있다. 그러나 에이전틱 AI가 더 많은 의사결정을 수행하고 다양한 도구를 호출하며 분산된 데이터 환경 전반에서 작동하기 시작하면, 가시성과 거버넌스, 신뢰가 약화될 수 있다.

SAS는 연례 행사 ‘SAS 이노베이트(SAS Innovate)’에서 이러한 문제에 대한 해법을 제시했다. 코파일럿, 에이전트 프레임워크, MCP(Model Context Protocol) 플러그인, 관리 도구 등 새로운 제품군을 공개하며, 기업이 통제력을 유지한 채 AI를 운영할 수 있도록 지원하겠다고 밝혔다.

SAS 글로벌 AI 및 생성형 AI 시장 전략 총괄 마리넬라 프로피는 해당 행사에서 “지금은 형태를 만드는 AI에서 실제로 행동하는 AI로 전환되는 시점”이라며 “이는 신뢰, 거버넌스, 책임성 측면에서 새로운 요구사항을 만들어내는 중요한 도약”이라고 말했다.

에이전트와 보다 직관적으로 상호작용

SAS는 먼저 바이야(Viya) 플랫폼에 내장된 대화형 AI 어시스턴트 ‘SAS 바이야 코파일럿’을 공개했다. 이 도구는 인간의 통제를 기반으로 작동하며, 마이크로소프트 파운드리(Microsoft Foundry)와 통합돼 분석 워크플로우 내에서 자연어를 활용해 데이터 분석, 모델 구축, 의사결정을 수행할 수 있도록 지원한다.

프로피는 “전문가 수준의 어시스턴트를 통해 사용자는 작업을 실행하고 질문을 던지며 전체 분석 라이프사이클을 보다 쉽게 탐색할 수 있다”고 설명했다.

바이야 코파일럿은 핵심 애플리케이션 전반에 걸친 질의응답, 설명 가능하고 문서화된 AI 코드 생성, 모델 파이프라인 가이드(추천 및 다음 단계 제시), 대화형 대시보드, AI 기반 검색과 알림 내러티브를 활용한 시각적 분석 기능 등을 제공한다. SAS에 따르면 향후 데이터 관리, 모델 관리, AI 인프라 영역까지 기능이 확장될 예정이다.

초기에는 두 가지 코파일럿이 먼저 제공된다. 자산·부채 관리(ALM)는 시나리오 개발과 금융 리스크 워크플로우 실행 및 해석, 자연어 입력을 분석 모델로 변환하는 기능을 지원한다. 헬스케어 임상 데이터 디스커버리는 데이터 분석, 코호트 생성, 연구 논문 및 의료 문서 탐색 기능을 제공한다.

SAS는 올해 말까지 은행, 제조 등 다양한 산업으로 바이야 코파일럿 적용 범위를 확대할 계획이다.

임베디드 AI 어시스턴트를 넘어, SAS는 내부 및 외부 에이전트를 연결하고 통제할 수 있는 도구와 인프라도 함께 제공한다. 새롭게 공개된 SAS 바이야 MCP 서버는 연결 방식을 표준화해, 클로드(Claude), GPT, 제미나이(Gemini) 등 다양한 대형언어모델(LLM)이나 인터페이스를 활용하는 외부 에이전트가 별도의 맞춤형 통합 없이도 SAS의 도구, 데이터, 모델에 안전하게 접근할 수 있도록 지원한다.

프로피는 “코파일럿은 단순히 질문에 답하는 것을 넘어, 바이야 전반의 기능을 보다 구조화된 방식으로 호출할 수 있다”고 말했다.

또한 SAS는 ‘에이전틱 AI 액셀러레이터’를 통해 코드, 인터페이스, 구성 요소, 모범 사례를 제공한다. 이를 통해 개발자뿐 아니라 로우코드·노코드 사용자까지 다양한 수준의 팀이 SAS 바이야 환경에서 에이전트를 설계하고 구축, 배포, 관리할 수 있다.

현재 바이야 사용자는 MCP 서버와 AI 액셀러레이터를 모두 깃허브를 통해 이용할 수 있다.

인간 판단 유지 위한 거버넌스 강화

SAS는 감독 체계와 신뢰 가능한 AI, 인간 개입 기반 통제의 중요성을 지속적으로 강조하고 있다. 이러한 전략의 일환으로 SaaS 기반 도구 ‘SAS AI 내비게이터’를 새롭게 공개했다.

이 도구는 기업이 보유한 AI 모델을 체계적으로 파악하고, 거버넌스를 적용하며, 정책을 일관되게 관리할 수 있도록 지원한다. 2026년 3분기 마이크로소프트 애저 마켓플레이스(Microsoft Azure Marketplace)를 통해 제공될 예정이며, 자체 개발 모델과 외부 모델을 포함해 기업 내 모든 AI 모델과 도구를 한눈에 파악할 수 있는 엔드투엔드 가시성을 제공한다. 이를 통해 기업은 내부 정책은 물론 외부 규제와 프레임워크까지 AI 활용 전반에 적용할 수 있다.

SAS 데이터 거버넌스 및 윤리 부문 부사장 레지 타운젠드는 “이 도구는 AI 자산에 대한 가시성을 제공할 뿐 아니라 ‘우리가 얼마나 잘하고 있는가’라는 근본적인 질문에도 답을 제시한다”고 말했다.

타운젠드는 기업이 평판, 효율성, 비용 등 다양한 요소를 동시에 고려하는 만큼, 이를 한눈에 파악할 수 있는 충분한 데이터가 필요하다고 강조했다. 이어 “신뢰는 이제 새로운 비즈니스 차별화 요소이자 일종의 통화로 인식되고 있다”고 덧붙였다.

그는 “책임 있는 AI를 자연스럽게 받아들이도록 만드는 것이 핵심”이라며, 기술 발전 속도와 조직의 적응 능력 간 격차를 의미하는 ‘기술 비대칭(tech asymmetry)’ 상황에서도 인간의 판단을 유지하기 위해 AI 거버넌스가 중요한 역할을 한다고 설명했다.

또한 “기술 역량은 충분하지만, 기업은 이를 대규모로 빠르게 적용하는 데 어려움을 겪고 있다”며 “이러한 역량을 지속 가능한 비즈니스 경쟁력으로 전환하는 것이 필요하다”고 강조했다.

AI 기능과 서비스가 빠르게 확장되는 상황에서, 타운젠드는 사용자들에게 충분한 이해도를 갖추고 호기심을 기반으로 AI를 접근하며, 기술이 비즈니스와 개인의 삶에 어떻게 적용될 수 있을지 비판적으로 사고할 것을 주문했다.

그는 “이처럼 빠르게 변화하는 환경에서는 확신을 잠시 내려놓을 필요가 있다”며 “확신은 경직성을 낳고, 이는 지금 필요한 섬세한 판단을 가로막을 수 있다”고 말했다. 이어 “AI의 다음 단계는 이러한 판단을 확장하고, 빠른 속도로 거버넌스를 적용하며, 신뢰를 경쟁력으로 전환하는 데 있다”고 강조했다.

신뢰 가능한 데이터 확보가 AI 성패 좌우

SAS는 기업 데이터 환경의 복잡성과 신뢰 문제도 주요 과제로 지목했다. SAS 산업 시장 총괄 앨리사 패럴은 온프레미스, 레거시 인프라, 프라이빗·퍼블릭 클라우드 등 다양한 환경에 데이터가 분산돼 있으며, 데이터 자체에 대한 신뢰 부족이 의사결정 신뢰 저하로 이어지고 있다고 설명했다. 또한 성능 제약 역시 AI 도입을 저해하는 요인으로 작용하고 있다고 지적했다.

이 문제를 해결하기 위해 SAS는 바이야 기반 클라우드 네이티브 데이터 관리 포트폴리오 ‘SAS 데이터 매니지먼트’를 개편했다. AI 활용을 고려한 데이터 관리, 설계 단계부터 적용되는 거버넌스, 에이전틱 AI 및 코파일럿 기능, 클라우드 기반 분석 가속 기능 등을 새롭게 추가하거나 강화했다.

패럴은 “이 플랫폼은 데이터가 접근되고 준비되며 활용되는 전체 워크플로우 내에서 데이터 계보, 투명성, 통제 기능을 제공한다”고 설명했다.

그는 “에이전트와 AI는 그 어느 때보다 더 많은 데이터를 필요로 한다”며 “특히 의사결정 자동화를 도입하는 경우, 초기 단계부터 데이터 관리 체계를 제대로 구축하는 것이 매우 중요하다”고 강조했다.

재설계된 플랫폼은 신뢰 가능한 데이터를 기반으로 AI를 구동하며, 원시 데이터를 AI 활용이 가능한 형태로 전환한다. 특히 클라우드 네이티브 분석 데이터 플랫폼 ‘스피디스토어(SpeedyStore)’를 통해 데이터를 이동하지 않고도 분석과 AI 처리를 수행할 수 있도록 했다. 이를 통해 대규모 데이터 이동 없이도 효율적인 처리가 가능해진다.

기업은 여전히 데이터 주권을 유지하면서 다양한 데이터 저장소 전반에 걸쳐 워크플로우를 통제할 수 있다.

패럴은 “고객이 현재 상황에 대응하는 데 필요한 모든 요소를 제공하고, 데이터를 활용하고 관리하며 가치를 창출할 수 있는 도구를 지원하고 있다”며 “이를 통해 기업은 자신 있게 AI를 대규모로 운영할 수 있다”고 말했다.
dl-ciokorea@foundryco.com

Enterprises are quickly moving from AI experimentation to deployment, however, when agentic AI begins making more decisions, invoking more tools, and operating across fragmented data environments, there can be an erosion of visibility, governance, and trust.

SAS laid out its answer to that problem at its annual conference, SAS Innovate, introducing a new family of copilots, agent frameworks, Model Context Protocol (MCP) plugins, and management tools to help enterprises operationalize AI without losing control of it.

“What we’re seeing here is really a shift from AI that forms to AI that acts,” Marinela Profi, the company’s global AI and generative AI market strategy lead, said at the event. “This is a significant leap, because it introduces new requirements around trust, around governance, around accountability.”

Interacting with agents more intuitively

To begin with, SAS today announced SAS Viya Copilot, a human-governed, conversational AI assistant embedded into its Viya platform. It integrates Microsoft Foundry, operating within analytics workflows to help developers, data scientists, and other users instructing it in natural language to analyze data, build models, and make decisions across workflows.

“You have an expert assistant that allows you to take actions, ask questions, and help you navigate across the full analytical lifecycle,” Profi explained.

Its capabilities include: General Q&A across core Viya applications; production of documented and explainable AI-generated code; model pipeline guidance including recommendations and next steps; conversational dashboarding; and visual investigation with AI-assisted search and alert narratives. Copilot capabilities will eventually extend to data management, model management, and AI infrastructure, according to SAS. 

The company is initially launching two Copilots: Asset and Liability Management (ALM), for developing scenarios, executing and interpreting financial risk workflows, and translating natural language inputs into analytic models; and Health Clinical Data Discovery, for analyzing data, creating cohorts, and investigating research papers and other medical documents.

SAS plans to expand Viya Copilot into additional industries, including banking and manufacturing, later this year.

Going beyond embedded AI assistants, SAS is providing tools and infrastructure to connect and govern internal and external agents. The new SAS Viya MCP server standardizes connections so external agents can safely access SAS tools, data, and models, using the large language model (LLM) or interface of their choice (Claude, GPT, Gemini), without having to create custom integrations, duplicate logic, or bypass controls.

“The Copilot is not only answering questions for you, it can invoke capabilities across Viya in a more structured way,” Profi said.

In addition, a new Agentic AI Accelerator provides a collection of code, interfaces, components, and best practices that allow teams across skill levels (developers, low-code or no-code users) to design, build, deploy, and manage agents within SAS Viya, she explained.

Current Viya users can access both the MCP server and AI Accelerator via GitHub.

Maintaining human judgment

SAS continues to emphasize the importance of oversight, trustworthy AI, and human-in-the-loop control.

Furthering this mission, the company is introducing SAS AI Navigator. The Software-as-a-Service (SaaS) tool helps enterprises inventory, govern, and apply policies to underlying AI models.

Available in Q3 2026 on Microsoft Azure Marketplace, the platform will offer an end-to-end view of all AI models and tools in use in an enterprise, whether built in-house or provided by third parties. Using it, enterprises will be able to apply internal policies and external regulations and frameworks to AI use cases.

“It’s giving visibility into your AI inventory,” Reggie Townsend, VP of SAS’ data governance and ethics practice, said at today’s event. “But it also answers the really basic question: How are we doing?”

Enterprises want “enough data at a glance” to consider tension points when they’re juggling factors like reputation, efficiency, and cost, he pointed out. They’re also viewing trust as a new business differentiator, even as a currency. 

Navigator started with a really simple idea, he noted: “What happens if we can make being responsible irresistible?” AI governance is one way to preserve human judgment amidst what he called “tech asymmetry.”

Technology unevenness has been a long-standing problem; While there’s strong technical capability, enterprises struggle to adapt to the pace of change at scale. “What folks need to do is try to translate some of these capabilities into a sustainable business advantage,” said Townsend.

As AI capabilities (and offerings) continue to expand, he urged users to gain “sufficient literacy,” approach AI with curiosity, and think critically about how evolving tools can apply to both business and personal life.

“In an emerging landscape like this, we’ve got to suspend certainty,” he said. “Certainty breeds rigidity, and rigidity suspends this idea of nuanced judgment, which we need right now.”

The next chapter of AI is about scaling that judgment, governing at speed, and turning trust into that competitive advantage, he emphasized.

Getting to the right enterprise data

Enterprise data can be fragmented across many different ecosystems (on-prem, in legacy infrastructure, or in private or public clouds), noted SAS industry market lead Alyssa Farrell. Beyond that, she said, “[enterprises] have low trust in the data itself, which is leading to low trust in decisions.” Further, performance constraints can hamper AI progress.

To address these issues, SAS today announced a targeted refresh of SAS Data Management, its cloud-native portfolio built on the Viya platform, adding or expanding its AI-ready data management, governance by design, agentic AI and copilots, and cloud-native analytic acceleration. It provides lineage, transparency, and control capabilities within workflows where data is accessed, prepared, and activated, Farrell explained.

“Agents and AI crave data more than ever before,” she said. “It’s really important that organizations get this right from the beginning, especially if they’re adding automation to that decision process.”

The re-architected platform grounds AI in trusted data, making raw data assets usable for AI. Notably, it brings analytics and AI to the data itself through SpeedyStore, the company’s cloud-native analytical data platform, negating the need to move volumes of data for processing, Farrell explained. Enterprises still retain digital sovereignty and can control workflows across their various data stores.

“We’re making sure our customers have everything they need to meet this moment [and] tools that access the data, manage the data and gain value from it,” Farrell noted. “They can really proceed at scale to operationalize AI with confidence.”

Most enterprise AI investments today focus on models, compute and tooling. The assumption is that intelligence is the binding constraint and that a more capable model will produce better outcomes across every dimension that matters. This is a reasonable starting point, but it is also where most initiatives go wrong.

The models organizations are deploying were trained on public data at scale. None of your internal systems, customer schema, pricing logic or support taxonomy appeared in that training.

When a model encounters your internal data, it processes it as best it can, but without the grounding that comes from having been trained on it. Early AI initiatives are struggling not because the models are weak, but because the context they need to operate reliably inside your organization is something they have never seen before.

Data engineering holds the key to this context.

Why context breaks first

Think about what an AI agent handling a support escalation needs to function well: The customer’s support history across time, not just the most recent ticket. Billing records matter too, because the character of a problem often depends on what the customer is paying for and whether anything has changed recently. Product usage data is equally essential, as what a customer reports is frequently explained by how they have been using the product. None of these things live in a single place, as they are scattered across systems that were each built by different teams, on different timelines, with different definitions of what a customer record is supposed to capture.

Human agents work around these gaps through judgment developed over time. They know which system to trust for a particular type of question, they know the usage data runs six hours behind and they know how to weigh conflicting signals based on context that is never written down anywhere. AI systems do not have that judgment. They process whatever they receive and act on it, which means that when the context is inconsistent or incomplete, the output reflects that, not as a visible error but as a subtly wrong decision. The customer notices before anyone on your team does.

When bad data stops being annoying and starts being operational

In the analytics era, data quality problems surfaced as numbers that looked off in dashboards. Analysts were the error-detection layer, and when something looked wrong, they would investigate, find the issue and get it fixed. The feedback loop was slow, but it existed, and it caught most problems before they reached the business in any consequential way.

AI agents making operational decisions do not have that buffer. They have no way of knowing that a schema migration introduced silent gaps or that a pipeline is running four hours late. Refunds go out incorrectly because the billing context was incomplete at the moment of decision.

What an analytics team could absorb as an occasional anomaly in a report becomes a real problem when an automated system acts on degraded context hundreds of times a day before anyone identifies the pattern. The volume is what makes it dangerous, and by the time it surfaces, the damage is already distributed across thousands of interactions.

The role data engineers play now

For the past decade, data engineering meant building pipelines that fed warehouses so analysts could query data and produce dashboards. The work was foundational but treated as background infrastructure, and its value was measured in pipeline reliability, query performance and reporting freshness.

The agent era changes the purpose of that work entirely. When AI systems make operational decisions, the goal is no longer producing data that is queryable. The goal is producing context that is reliable enough for a system to act on, and those are different problems with different requirements. That starts with entity resolution across systems, providing a consistent and trustworthy answer across every data source that touches them.

This also means handling late-arriving data explicitly, because agents cannot act on a state of the world that no longer holds. Freshness thresholds need to be calibrated to the decision type, since a personalization recommendation can tolerate six-hour-old usage data in ways that a refund workflow cannot. Lineage needs to survive schema changes and reorganizations, so that the provenance of any piece of context can be traced when something goes wrong.

None of that is a model problem, nor does it yield to prompt engineering. This is data engineering work, and organizations that treat it as anything else will spend a long time debugging production failures that look like AI problems but are infrastructure problems.

Context is only half the problem

Getting the right information to an agent is necessary, but it is not sufficient. There is a second challenge that most organizations have not yet confronted: How do you coordinate, govern and operate dozens or hundreds of autonomous agents making real decisions across your business?

Agent frameworks handle reasoning well. What they do not handle is everything around the agent: Scheduling when it runs, controlling what it is allowed to spend, enforcing who can approve its decisions, managing retries when external systems fail and ensuring that when an agent needs human sign-off, it does not tie up compute for hours while it waits. These are not AI problems. They are operational infrastructure problems, and they are the same class of problems that orchestration platforms have been solving for data pipelines for over a decade.

One agent answering questions in a sandbox is a proof of concept. Fifty agents making operational decisions across finance, compliance and customer operations is a fleet management problem, and it requires the same kind of scheduling, governance, cost controls and auditability that enterprises already demand from their data infrastructure.

Orchestration is typically the one layer that already has visibility across platforms, spanning your warehouse, your transformation layer, your external APIs and your operational databases. That cross-platform vantage point is what makes it possible to build a context layer that is comprehensive rather than siloed.

Governance needs to execute at runtime, not live in documentation. Policies about data access, cost limits and human approval requirements need to be enforced in code as agents run, not described in guidelines that agents cannot read and humans forget to follow.

What this means going forward

The organizations that deploy AI agents at scale will have invested in two things before those agents reach production.

First, a context layer that gives agents a reliable, cross-platform understanding of the enterprise’s data. This means not just raw access to tables, but semantic knowledge of what the data means, where it comes from and how much to trust it.

Second, an operational layer that governs how agents act, with the scheduling, cost controls, auditability and human-in-the-loop checkpoints that enterprise deployment demands.

These two investments are not independent. They form a flywheel. Better context makes agents more effective, which drives broader adoption, which generates richer operational metadata, which deepens the context layer further.

Data engineers are becoming the people who determine whether automated decisions are trustworthy, not because they control the models but because they control both the context on which those models operate and the infrastructure through which they act. The organizations that understand this early will keep building on it. The ones that keep treating data engineering and orchestration as background infrastructure will keep rediscovering the same production failures, just with different names on the postmortem each time.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?

“Data is the new oil” is one of the most overused phrases in enterprise technology. Yet it still captures something fundamentally true about the modern enterprise, if we extend the analogy.

Crude oil has limited value until it is refined into the fuels, chemicals, plastics, polymers, synthetic fibers, and industrial materials that power entire societies and permeate nearly every aspect of modern life. Similarly, the real value of data does not lie in its raw accumulation but in its transformation, through systems, into decisions, intelligence, and operational impact.

In this context, converged analytics has emerged as the refinery of the data economy. Organizations that lead will be those with the most effective refining layer. 

Traditional analytics architectures evolved in silos, no longer compatible with the dynamic AI world

Over the past decade, enterprises have invested heavily in extracting, storing, and moving data. Data lakes, warehouses, streaming platforms, and cloud pipelines have created an unprecedented accumulation of information. And yet only 13% of enterprises globally are successfully achieving ROI from their AI initiatives. 

“Enterprises now sit on massive reserves of structured, semi-structured, and unstructured data generated by applications, devices, and digital interactions. Yet despite this abundance, many CIOs still struggle to translate data into consistent, real-time business value. The issue is not scarcity—it is fragmentation,” says Quais Taraki, CTO, EnterpriseDB (EDB). 

The value of data is trapped when it’s siloed and spread across systems and teams. 

Transactional systems were optimized for operational workloads. Analytical systems were built for reporting and historical analysis. Streaming systems handled real-time events. Each requires different infrastructure, tools, and governance models. Data has to be copied, moved, transformed, and reconciled across environments before it can be used. This introduces latency, complexity, duplication, and risk. Insights often arrive too late to influence outcomes, while operational systems remain disconnected from analytical intelligence.

Converged analytics solves the largest challenge for AI-ready data

What makes crude oil valuable is not extraction alone but its combination with the refinery—the integrated industrial system that processes, synthesizes, and upgrades raw hydrocarbons into usable products. 

Comparable in the world of enterprise technology is converged analytics, which addresses data systems fragmentation by unifying capabilities into a single, sovereign architectural paradigm. It brings together transactional processing, analytical processing, and streaming-data handling within a cohesive system. 

“Instead of moving data across multiple specialized platforms, converged analytics enables computation to occur where the data resides, across different workloads and time horizons. This integration collapses latency, reduces duplication, and preserves context, allowing organizations to move from retrospective analysis to real-time decision-making,” says Taraki of EDB.

AI raises the stakes 

While generative AI and now agentic AI have captured executive attention, their effectiveness depends on access to fresh, well-governed, and contextually rich data. Models trained on stale or fragmented datasets deliver limited value. 

Converged analytics provides the foundation for continuous data pipelines, real-time feature engineering, and low-latency inference. It enables architectures such as retrieval-augmented generation and supports ongoing feedback loops that improve model performance over time. In this sense, it is not just complementary to AI; it is a prerequisite for operationalizing it at scale.

AI also intensifies the cost of fragmentation. 

“Every time data must be copied, moved, or reconciled across specialized systems, organizations introduce latency, duplication, and loss of context,” says Taraki. 

Converged analytics reduces that friction by enabling computation closer to where data already resides, allowing decisions to happen in real time rather than after the fact.

Converged analytics offers non-AI and data companies a pathway to increased relevance and value

Unlike point solutions that address isolated parts of the data pipeline, converged analytics platforms sit at the center of the entire data lifecycle. They intersect with storage, compute, networking, and security, making them a natural integration point for a wide range of technologies. 

For hardware vendors, this creates demand for high-performance infrastructure capable of handling mixed workloads with low latency and high throughput. For service providers, it opens the door to long-term engagements around platform design, deployment, optimization, and governance.

Converged analytics workloads are not peripheral use cases; they are core to business performance. Real-time fraud detection, predictive maintenance, personalized customer experiences, and supply chain optimization all depend on the ability to process and act on data as it is generated. These workloads are both compute intensive and mission critical, making converged analytics an especially valuable category for vendors seeking to align with enterprise priorities.

The shift toward hybrid and edge computing environments adds another dimension to the opportunity. As enterprises distribute workloads across cloud, on-premises, and edge locations, the need for consistent analytics capabilities across these environments becomes critical. 

Converged analytics platforms are increasingly designed to operate seamlessly across this spectrum, enabling data to be processed and acted upon wherever it is generated. This creates additional insertion points for both hardware and services vendors, from edge devices and accelerators to orchestration, lifecycle management, and ongoing operational support.

Making it work at enterprise scale

In the early stages of the oil industry, value was concentrated in extraction. Over time, it shifted to refining and distribution, with efficiency, scale, and integration determining competitive advantage. The same transition is now underway in the data economy. Enterprises already possess vast reserves of data; the differentiator will be their ability to refine it rapidly, efficiently, and in context.

Converged analytics represents that refining capability. It is why hardware vendors are optimizing for data-intensive workloads and why services firms are reorganizing around platform engineering. But the practical reality is that this refining layer cannot succeed as software alone. It depends on the hardware, services, support, and operational expertise required to deploy and run it at scale.

For CIOs, this is no longer just a question of architecture. It is a prerequisite for making data a true driver of business value. To learn more, visit us here.

Google is recasting its data and analytics portfolio as the Agentic Data Cloud, an architecture it says is aimed at moving enterprise AI from pilot to production by turning fragmented data into a unified semantic layer that agents can reason over and act on more reliably at scale.

The new architecture builds on Google’s existing data platform strategy, bringing together services such as BigQuery, Dataplex, and Vertex AI, and elevating their capabilities in metadata, governance, and cross-cloud interoperability into what the company describes as a shared intelligence layer.

That intelligence layer strategy is underpinned by the new Knowledge Catalog, an evolution of Dataplex Universal Catalog, that the company said uses new capabilities to extend its metadata foundation into a semantic layer mapping business meaning and relationships across data sources.

These capabilities include native support for third-party catalogs, applications such as Salesforce, Palantir, Workday, SAP, and ServiceNow, and the option to move third-party data to Google’s lakehouse, which automatically maps the data to Knowledge Catalog.

To capture business logic more directly for data stored inside Google Cloud, the company is adding tools including a LookML-based agent, currently in preview, that can derive semantics from documentation, and a new feature in BigQuery, also in preview, that allows enterprises to embed that business logic for faster data analysis.

Beyond aggregation, the catalog itself is designed to continuously enrich semantic context by analyzing how data is used across an enterprise, senior google executives wrote in a blog post.

This includes profiling structured datasets as well as tagging and annotating unstructured content stored in Google Cloud Storage, the executives pointed out, adding that the catalog’s underlying system can also infer missing structure in data by  using its Gemini models to generate schemas and identify relationships.

Turning data into business context the next battleground for AI

For analysts, Google’s focus on semantics targets one of the biggest barriers to production AI for enterprises.

“The hardest AI problem is inconsistent meaning,” said Dion Hinchcliffe, lead of the CIO practice at The Futurum Group, noting that a unified semantic layer could help CIOs establish consistent business context across systems while reducing the need for developers to manually stitch together metadata and lineage.

That focus on semantic context also reflects a broader shift in how hyperscalers are approaching enterprise AI. Microsoft with Fabric IQ and AWS with Nova Forge are pursuing similar strategies, building semantic context layers over enterprise data to make AI systems more consistent and easier to operationalize at scale.

While Microsoft’s approach is to wrap AI applications and agents with business context and semantic intelligence in its Fabric IQ and Work IQ offerings, AWS want enterprises to blend business context into a foundational LLM by feeding it their proprietary data.

Mike Leone, principal analyst at Moor Insights and Strategy, said Google’s approach, though closer to Microsoft’s, places the data gravity one layer above the lakehouse, within its data catalog and semantic graph capabilities.

“Google and Microsoft are solving the same problem from different angles, Fabric through a unified data foundation and Google through a unified semantic and context layer,” Leone said.

Even data analytics software vendors are converging on the idea of offering a catalog that can map semantic context from a variety of data sources, Leone added, pointing to Databricks’ Unity Catalog and Snowflake’s Horizon Catalog.

Semantic accuracy could pose challenges for CIOs

However, Google’s approach to building an intelligent semantic layer, especially its evolved Knowledge Catalog, comes with its own set of risks for CIOs.

The new catalog’s automated semantic context refinement capability, according to Jim Hare, VP analyst at Gartner, could amplify governance challenges, especially around metadata management: “In complex enterprise domains, errors in inferred relationships or definitions will require ongoing human domain oversight to maintain trust.”

Hare also warned of operational and cost management challenges.

“Agent-driven workflows spanning analytical and operational data, potentially across clouds, will introduce new challenges in observability, debugging, and cost predictability,” he said. “Dynamic agent behavior can generate opaque consumption patterns, requiring chief data and analytics officers (CDAOs) to closely manage cost attribution, usage limits, and operational guardrails as these capabilities mature.”

Adopting Google’s new architectural approach could increase dependence at the orchestration layer, resulting in issues around portability, he warned: “Exiting Google-managed semantics, Gemini agents, or BigQuery abstractions may be harder than migrating data alone.”

Bi-directional federation as strategic play

Even so, the trade-offs may be acceptable for enterprises prioritizing tighter data integration over flexibility.

As part of the new architecture, Google is also offering cross-platform data interoperability via the Apache Iceberg REST Catalog that it says will allow bi-directional federation, in turn letting enterprises access, query, and govern data across environments such as Databricks, Snowflake, and AWS without requiring data movement or cost in egress fees.

For Stephanie Walter, practice leader of the AI stack at HyperFRAME Research, this interoperability will be strategically important for enterprises scaling agents in production, especially ones that have heterogenous data environments.

Moor Insights and Strategy’s Leone, though, sees it as a different strategic play to address enterprises’ demand to access Databricks, Snowflake, and hyperscaler environments without costly data movement.

Google’s Agentic Data Cloud architecture also includes a Data Agent Kit, currently in preview, which the company says is designed to help enterprises build, deploy, and manage data-aware AI agents that can interact with governed datasets, apply business logic, and execute workflows across systems.

Robert Kramer, managing partner at KramerERP, said the Data Agent Kit will help data practitioners abstract t daily tasks, in turn lowering the barrier to operationalizing agentic AI across workflows.

However, Gartner’s Hare warned that enterprises should guard against over delegating critical data management decisions to automated agents without sufficient observability, validation controls, and human review, particularly where downstream AI systems depend on these agents for continuous data operations.

This article first appeared on InfoWorld.

Snowflake is enhancing Snowflake Intelligence and Cortex Code to create a unified experience connecting enterprise systems, data sources, and AI models with Snowflake data. It’s part of the company’s vision to become the control plane for the agentic enterprise, enabling enterprises to align data, tools, and workflows with AI agents built on its platform.

With these updates, the company said, Snowflake Intelligence becomes an adaptable personal work agent for business users, and Cortex Code expands as a builder layer for enterprise AI that provides governed, data-native development.

Enhancements to Snowflake Intelligence include automation of routine tasks by describing them in natural language, new Model Context Protocol (MCP) connectors, and reusable artifacts that let users save and share analyses, visualizations, and workflows, all of which will be generally available “soon.” In addition, a new iOS mobile app, and multi-step reasoning with deep research that uses agentic architecture to reason across data will soon be in public preview.

The company said that all of these updates came out from customer feedback, as well as from insights gleaned from Project SnowWork, last month’s preview of an autonomous AI layer for its data cloud.

Cortex Code now supports additional external data sources, including AWS Glue, Databricks, and Postgres, connectivity with other AI agents via MCP and Agent Communication Protocol (ACP), a Claude Code plugin, and a new agent software development kit with support for Python and TypeScript. There are also enhancements to Cortex Code in Snowsight, Snowflake’s web interface, including Plan Mode to allow developers to preview and approve workflows, and Snap & Ask to enable interaction with data artifacts such as charts and tables.

Snowflake also announced the private preview of Cortex Code Sandboxes in Snowsight, a dedicated cloud environment where developers can execute code end-to-end with no setup.

Michael Leone, VP & principal analyst at Moor Insights & Strategy, thinks the roadmap is “ambitious,” noting the number of items announced that are “coming soon” or are in public preview. “These announcements are starting to blur together, with almost every vendor claiming their agents can reason, act, and transform the business,” he said, adding, “What makes this one worth slowing down on, at least for me, is that Snowflake is going after both halves of the enterprise at the same time. Intelligence is built for the business users who want answers and actions without writing SQL, and Cortex Code is built for the builders who actually have to put this into production.”

Most vendors pick one target, users or builders, and come back to the other later, he said, but Snowflake is putting both on the same governed data foundation. “[This] is a harder engineering problem, but I’d argue it’s a cleaner answer to the question enterprises are actually asking, which is how to open AI up to more people without losing control of the data underneath,” he said, noting that Snowflake has changed its approach from “let’s do it inside Snowflake,” to realizing that agentic AI only works if it’s interoperable with the rest of the stack.

Igor Ikonnikov, advisory fellow at Info-Tech Research Group, also sees the control plane play as part of an industry trend. “As always, the devil is in the details: what those platforms are composed of and how they offer to control AI agents,” he said. “Most platforms are built the old-fashioned way: All the controls are coded. Snowflake speaks about reusable analytics through saving the whole solution and reusing complete modules or models. It means that common semantics are still buried inside database models and code.”

All AI vendors are motivated by the same demand from the market, he said: “Move from Copilot-based generic chatbots to business-purpose-specific AI agents that understand business logic and can interact with one another.” With these updates, he sees Snowflake as having caught up with the competition, but not yet surpassing it.

Sanjeev Mohan, principal at SanjMo, said, “The good news for customers is the support for Databricks and AWS Glue. What Snowflake is saying is that even if your data lives in a competitor’s system, Snowflake AI coding agent can be used. And vice versa, the VS Code extension and Claude Code plugin can be used on Snowflake data. In other words, it reduces vendor lock-in fears.”

It’s also the right strategic direction, said Sanchit Vir Gogia, chief analyst at Greyhound Research. “Enterprise AI is moving from generation to orchestration to execution, and Snowflake’s focus on governed data as the foundation for action aligns with that shift,” he said.

“However, becoming the execution layer for enterprise AI requires more than integrating agents and expanding tooling,” he said. It also requires consistent semantics, reliable cross-system execution, strong governance, economic viability, and organisational readiness, as well as overcoming a structural constraint. “Control without ownership of the systems where work is executed introduces dependency that is difficult to fully resolve. This is the central tension in Snowflake’s strategy and will define how far it can realistically extend its influence,” he said. “Snowflake has taken a meaningful step in that direction. It has not yet proven that it can deliver this at scale. At this stage, it is one of the most credible contenders in a race that will be defined not by who builds the smartest AI, but by who can make that AI work reliably inside the enterprise.”

This article first appeared on InfoWorld.

Revenue dips. Latency spikes. Alerts fire. The dashboards look fine – until they don’t

Slack explodes. Ten engineers become 20. Queries multiply. Everyone starts scanning raw event data at once. And then the system starts to buckle. Right when you need it most.

Over the past decade, I’ve worked on large-scale, real-time analytics systems for massive, bursty workloads. First in ad tech and more recently in observability. Across very different domains, the same failure pattern tends to emerge. Platforms that perform well under normal, steady-state conditions degrade under investigative load.

In many cases, this isn’t simply a matter of tuning or operational discipline. It reflects architectural assumptions. Most observability platforms were designed for detection-oriented workloads and not the unpredictable, exploratory way humans investigate incidents in real time.

Where the architecture breaks

Many observability platforms are built around a core assumption that queries will follow normal, predictable patterns. Dashboards, alerts and saved searches reflect known questions about the system.

But incidents aren’t predictable.

During an investigation, workloads shift instantly. Queries become exploratory. Time ranges expand. Filters change constantly. Concurrency spikes as multiple teams dig into the same data.

Architectural assumptions that work well in steady state can begin to show strain. Index-centric systems perform well on known paths. Step outside them, and performance drops quickly. Sub-second queries turn into minutes, concurrency falls off and costs rise.

Over time, teams may begin to limit the scope of analysis or to export data to other systems simply to maintain responsiveness.

This dynamic isn’t primarily about features. It reflects a structural mismatch between how many systems are designed and how investigations actually unfold.

What “event-native” actually means

Over the past decade, several large-scale real-time analytics systems — including Apache Druid, something I’ve been intimately engaged with — were designed to handle highly bursty, event-driven workloads.

These environments required a different architectural model.

Rather than optimizing around predefined views or tightly coupled indexing structures, event-native systems treat raw, immutable events as the primary unit of storage and analysis. Every request, error and interaction is preserved as an event and remains available for exploration.

Data is stored in column-oriented formats designed for large-scale scanning and high-cardinality queries. Instead of shaping the data upfront for specific access patterns, the system is built to support evolving questions directly against the event stream.

The difference becomes clear during an incident.

Imagine a latency spike affecting a subset of users. Engineers may need to pivot across user ID, region, service version or request path — combining dimensions that were not anticipated in advance.

In an event-native system, those pivots can occur directly against stored event data without rebuilding indexes or reshaping datasets for each new question. Multiple teams can run these queries concurrently, even across large time ranges, without the system degrading.

That’s the core shift: you’re no longer constrained by how the data was modeled upfront. You can investigate what actually happened, in real time, at scale.

Cloud economics changed the rules, but architectures stayed the same

Many observability architectures were designed for an era when storage was fixed (and expensive). That’s no longer the case. In the cloud, storage is abundant and cheap. Compute is elastic, which is often the real cost driver. You can store years of event data in object storage at a fraction of the cost of running always-on compute clusters. Yet many observability platforms still tightly couple storage, indexing and query compute as if nothing changed.

What does this mean in practice? You pay peak compute prices just to keep data available and accessible. This turns observability into making bad trade-offs between cost, retention and performance.

All-in-one observability platforms can be powerful, but they’re also rigid. When storage and compute scale together, you lose control over economics.

Monolithic architectures shine in steady state, but when incidents are triggered, they quickly become painfully expensive, painfully slow or both.

Why observability needs a dedicated data layer, not another all-in-one platform

For years, consolidation has been a common response in observability – one more all-in-one platform promising simplicity.

That approach can reduce surface complexity in the short term. Over time, however, tightly coupled systems can limit flexibility. As scale increases, storage, compute and visualization begin to compete for resources inside the same architecture.

Business intelligence learned this lesson decades ago. What started as tightly coupled stacks separated into a modular architecture where storage, transformation and visualization became independent layers. That separation created leverage and companies like Snowflake, Databricks, Fivetran and Tableau emerged by focusing on distinct parts of the stack.

Each layer could innovate independently. Storage could scale without changing dashboards or workflow, compute engines could evolve without changing ingestion and visualization tools could compete on experience rather than infrastructure.

Observability is next.

One architectural response is the introduction of a purpose-built data layer that sits beneath existing observability tools such as Splunk, Grafana or Kibana. By separating data storage from interaction and analysis, organizations can retain large volumes of telemetry while scaling compute based on investigative demand.

It means longer retention without constant peak compute costs. It means bursty, investigative workloads don’t collapse the system and multiple teams can dig into the same event stream without stepping on each other. It aligns the architecture with how observability admins and engineers actually work during incidents.

And critically so, it treats observability as a data infrastructure problem not just a tooling problem.

This shift breaks the lock between data and tools

In tightly integrated observability platforms, data is often bound to a specific query engine or user interface. That coupling can simplify adoption, but it also limits long-term flexibility. Storage decisions, retention policies and performance characteristics become tied to a single vendor’s architecture.

When the underlying event data layer is open, durable and scalable, organizations gain optionality. The same telemetry can be analyzed across multiple tools. Retention strategies can evolve independently of dashboards. New query engines or visualization systems can be adopted over time without migrating years of historical data.

That’s why new architectural patterns are emerging in large-scale deployments – systems designed for unpredictable query shapes and deep exploratory analysis. Architectures that separate storage, compute and indexing that treat observability as a data problem first.

When data is stored in open, scalable systems rather than locked inside a single platform, organizations gain flexibility. They can analyze the same data across multiple tools, adopt new technologies over time and avoid being constrained by the limitations or cost structures of any one vendor.

What the next decade of observability will look like

Telemetry volumes will continue to grow. Distributed systems introduce more surface area. AI workloads generate additional signals and amplify data scale. Investigations are becoming more collaborative and more exploratory.

In that environment, the defining characteristic of observability systems will not be the number of features they expose, but the architecture beneath them.

When Slack explodes and dashboards slow down with (or completely stop) answering the right questions, the architecture underneath will determine whether teams find the root cause in minutes or watch the system buckle all over again.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?