What is data analytics?

Data analytics focuses on gleaning insights from data. It comprises the processes, tools, and techniques of data analysis and management, and its chief aim is to apply statistical analysis and technologies on data to find trends and solve problems. Data analytics has become increasingly important in the enterprise to shape business processes and improve decision-making and business results.

Data analytics draws from a range of disciplines, including computer programming, mathematics, and statistics, to perform analysis on data in an effort to describe, predict, and improve performance. So to ensure robust analysis, data analytics teams leverage a range of data management techniques, including data mining, data cleansing, data transformation, data modeling, and more.

What is AI data analytics?

AI data analytics is a rapidly growing specialty within data analytics that applies AI to support, automate, and simplify data analysis. It leverages ML, natural language processing, and data mining, along with foundational models and chat assistance for predictive analytics, sentiment analysis, and AI-enhanced business intelligence. AI tools can be used for data collection and data preparation, while ML models can be trained to extract insights and patterns.

The four types of data analytics

Analytics breaks down broadly into four types: descriptive analytics attempts to describe what has transpired at a particular time; diagnostic analytics assesses why something has happened; predictive analytics ascertains the likelihood of something happening in the future; and prescriptive analytics provides recommended actions to take to achieve a desired outcome.

To explore these more specifically, descriptive analytics uses historical and current data from multiple sources to describe the present state, or a specified historical state, by identifying trends and patterns. Business analytics is the purview of business intelligence (BI). Diagnostic analytics uses data, often generated via descriptive analytics, to discover the factors or reasons for past performance. Predictive analytics applies techniques such as statistical modeling, forecasting, and ML to the output of descriptive and diagnostic analytics to make predictions about future outcomes. Predictive analytics is often considered a type of advanced analytics, and frequently depends on ML and/or deep learning. And prescriptive analytics is another type of advanced analytics that involves the application of testing and other techniques to recommend specific solutions that will deliver outcomes. In business, predictive analytics uses ML, business rules, and algorithms.

Data analytics methods and techniques

Data analysts use a number of methods and techniques to analyze data. According to Emily Stevens, managing editor at CareerFoundry, seven of the most popular include:

1. Regression analysis: A set of statistical processes used to estimate the relationships between variables to determine how changes to one or more might affect another, like how social media spending might affect sales.
2. Monte Carlo simulation: A mathematical technique, frequently used for risk analysis, that relies on repeated random sampling to determine the probability of various outcomes of an event that can’t otherwise be readily predicted due to degrees of uncertainty in its inputs.
3. Factor analysis: A statistical method for taking a massive data set and reducing it to a smaller, more manageable one to uncover hidden patterns, like when analyzing customer loyalty.
4. Cohort analysis: A form of analysis in which a dataset is broken into groups that share common characteristics, or cohorts, for analysis like understanding customer segments.
5. Cluster analysis: A statistical method in which items are classified and organized into clusters in an effort to reveal structures in data. Insurance firms might use cluster analysis to investigate why certain locations are associated with particular insurance claims, for instance.
6. Time series analysis: A statistical technique in which data in set time periods or intervals is analyzed to identify trends over time, such as weekly sales numbers or quarterly sales forecasting.
7. Sentiment analysis: A technique that uses natural language processing, text analysis, computational linguistics, and other tools to understand sentiments expressed in data, such as how customers feel about a brand or product based on responses in customer forums. While the previous six methods seek to analyze quantitative or measurable data, sentiment analysis seeks to interpret and classify qualitative data by organizing it all into themes.

Data analytics tools

Data analysts use a range of tools to aid them surface insights from data. Some of the most popular include: 

• Apache Spark: An open source data science platform to process big data and create cluster computing engines. 
• AskEnola AI: A conversational analytics tool for business users.
• Data analysis with ChatGPT: OpenAI’s chatbot can generate code to perform data analysis, transformation, and visualization tasks using Python.
• dbt: An open source analytics engineering tool for data analysts and engineers.
• Domo Analytics: A BI SaaS platform to gather and transform data.  
• Excel: Microsoft’s spreadsheet software for mathematical analysis and tabular reporting. 
• Julius AI: An AI assistant to analyze spreadsheets and databases.
• Knime: A free and open source data cleaning and analysis tool for data mining.
• Looker: Google’s data analytics and BI platform. 
• MySQL: An open source relational database management system to store application data used in data mining.
• Observable: A data analysis platform with AI tools for exploratory data analysis and data visualization.
• Orange: A data mining tool ideal for smaller projects.
• Power BI: Microsoft’s data visualization and analysis tool to create and distribute reports and dashboards. 
• Python: An open source programming language popular among data scientists to extract, summarize, and visualize data. 
• Qlik: A suite of tools to explore data and create data visualizations. 
• R: An open source data analytics tool for statistical analysis and graphical modeling. 
• RapidMiner: A data science platform that includes a visual workflow designer. 
• SAS: An analytics platform for business intelligence and data mining. 
• Sisense: A popular self-service BI platform. 
• Tableau: Data analysis software from Salesforce to create data dashboards and visualizations.

Data analytics vs. data science

Data analytics is a component of data science used to understand what an organization’s data looks like. Generally, the output of data analytics are reports and visualizations. Data science takes the output of analytics to study and solve problems. The difference between data analytics and data science is often about timescale. Data analytics describes the current or historical state of reality, whereas data science uses that data to predict and/or understand the future.

Data analytics vs. data analysis

While the terms data analytics and data analysis are frequently used interchangeably, data analysis is a subset of data analytics concerned with examining, cleansing, transforming, and modeling data to derive conclusions. Data analytics includes the tools and techniques used to perform data analysis.

Data analytics vs. business analytics

Business analytics is another subset of data analytics. It uses data analytics techniques, including data mining, statistical analysis, and predictive modeling, to drive better business decisions. Gartner defines business analytics as solutions used to build analysis models and simulations to create scenarios, understand realities, and predict future states.

Data analytics examples

Organizations across all industries leverage data analytics to improve operations, increase revenue, and facilitate digital transformations. Here are three examples:

UPS transforms air cargo operations with data, AI: UPS’s Gateway Technology Automation Platform (GTAP) uses AI and digital asset tracking to reduce costs, improve on-time performance, and enhance operational safety at its Worldport air hub.

NFL leverages AI and predictive analytics to reduce injuries: The NFL’s Digital Athlete platform leverages AI and ML to run millions of simulations of in-game scenarios, using video and player tracking data to identify the highest risk of injury during plays, and develop individualized injury prevention courses.

Fresenius Medical Care anticipates complications with predictive analytics: Fresenius Medical Care, which specializes in providing kidney dialysis services, is pioneering the use of a combination of near real-time IoT data and clinical data to predict when kidney dialysis patients might suffer a potentially life-threatening complication called intradialytic hypotension (IDH).

Data analytics salaries

According to data from PayScale, the average annual salary for a data analyst is $70,384, with a reported range from $51,000 to $95,000. Salary data on similar positions include:

PayScale also identifies cities where data analysts earn salaries that are higher than the national average. These include San Francisco (24.2%), Seattle (10.2%), and New York (9.5%).

Revenue dips. Latency spikes. Alerts fire. The dashboards look fine – until they don’t

Slack explodes. Ten engineers become 20. Queries multiply. Everyone starts scanning raw event data at once. And then the system starts to buckle. Right when you need it most.

Over the past decade, I’ve worked on large-scale, real-time analytics systems for massive, bursty workloads. First in ad tech and more recently in observability. Across very different domains, the same failure pattern tends to emerge. Platforms that perform well under normal, steady-state conditions degrade under investigative load.

In many cases, this isn’t simply a matter of tuning or operational discipline. It reflects architectural assumptions. Most observability platforms were designed for detection-oriented workloads and not the unpredictable, exploratory way humans investigate incidents in real time.

Where the architecture breaks

Many observability platforms are built around a core assumption that queries will follow normal, predictable patterns. Dashboards, alerts and saved searches reflect known questions about the system.

But incidents aren’t predictable.

During an investigation, workloads shift instantly. Queries become exploratory. Time ranges expand. Filters change constantly. Concurrency spikes as multiple teams dig into the same data.

Architectural assumptions that work well in steady state can begin to show strain. Index-centric systems perform well on known paths. Step outside them, and performance drops quickly. Sub-second queries turn into minutes, concurrency falls off and costs rise.

Over time, teams may begin to limit the scope of analysis or to export data to other systems simply to maintain responsiveness.

This dynamic isn’t primarily about features. It reflects a structural mismatch between how many systems are designed and how investigations actually unfold.

What “event-native” actually means

Over the past decade, several large-scale real-time analytics systems — including Apache Druid, something I’ve been intimately engaged with — were designed to handle highly bursty, event-driven workloads.

These environments required a different architectural model.

Rather than optimizing around predefined views or tightly coupled indexing structures, event-native systems treat raw, immutable events as the primary unit of storage and analysis. Every request, error and interaction is preserved as an event and remains available for exploration.

Data is stored in column-oriented formats designed for large-scale scanning and high-cardinality queries. Instead of shaping the data upfront for specific access patterns, the system is built to support evolving questions directly against the event stream.

The difference becomes clear during an incident.

Imagine a latency spike affecting a subset of users. Engineers may need to pivot across user ID, region, service version or request path — combining dimensions that were not anticipated in advance.

In an event-native system, those pivots can occur directly against stored event data without rebuilding indexes or reshaping datasets for each new question. Multiple teams can run these queries concurrently, even across large time ranges, without the system degrading.

That’s the core shift: you’re no longer constrained by how the data was modeled upfront. You can investigate what actually happened, in real time, at scale.

Cloud economics changed the rules, but architectures stayed the same

Many observability architectures were designed for an era when storage was fixed (and expensive). That’s no longer the case. In the cloud, storage is abundant and cheap. Compute is elastic, which is often the real cost driver. You can store years of event data in object storage at a fraction of the cost of running always-on compute clusters. Yet many observability platforms still tightly couple storage, indexing and query compute as if nothing changed.

What does this mean in practice? You pay peak compute prices just to keep data available and accessible. This turns observability into making bad trade-offs between cost, retention and performance.

All-in-one observability platforms can be powerful, but they’re also rigid. When storage and compute scale together, you lose control over economics.

Monolithic architectures shine in steady state, but when incidents are triggered, they quickly become painfully expensive, painfully slow or both.

Why observability needs a dedicated data layer, not another all-in-one platform

For years, consolidation has been a common response in observability – one more all-in-one platform promising simplicity.

That approach can reduce surface complexity in the short term. Over time, however, tightly coupled systems can limit flexibility. As scale increases, storage, compute and visualization begin to compete for resources inside the same architecture.

Business intelligence learned this lesson decades ago. What started as tightly coupled stacks separated into a modular architecture where storage, transformation and visualization became independent layers. That separation created leverage and companies like Snowflake, Databricks, Fivetran and Tableau emerged by focusing on distinct parts of the stack.

Each layer could innovate independently. Storage could scale without changing dashboards or workflow, compute engines could evolve without changing ingestion and visualization tools could compete on experience rather than infrastructure.

Observability is next.

One architectural response is the introduction of a purpose-built data layer that sits beneath existing observability tools such as Splunk, Grafana or Kibana. By separating data storage from interaction and analysis, organizations can retain large volumes of telemetry while scaling compute based on investigative demand.

It means longer retention without constant peak compute costs. It means bursty, investigative workloads don’t collapse the system and multiple teams can dig into the same event stream without stepping on each other. It aligns the architecture with how observability admins and engineers actually work during incidents.

And critically so, it treats observability as a data infrastructure problem not just a tooling problem.

This shift breaks the lock between data and tools

In tightly integrated observability platforms, data is often bound to a specific query engine or user interface. That coupling can simplify adoption, but it also limits long-term flexibility. Storage decisions, retention policies and performance characteristics become tied to a single vendor’s architecture.

When the underlying event data layer is open, durable and scalable, organizations gain optionality. The same telemetry can be analyzed across multiple tools. Retention strategies can evolve independently of dashboards. New query engines or visualization systems can be adopted over time without migrating years of historical data.

That’s why new architectural patterns are emerging in large-scale deployments – systems designed for unpredictable query shapes and deep exploratory analysis. Architectures that separate storage, compute and indexing that treat observability as a data problem first.

When data is stored in open, scalable systems rather than locked inside a single platform, organizations gain flexibility. They can analyze the same data across multiple tools, adopt new technologies over time and avoid being constrained by the limitations or cost structures of any one vendor.

What the next decade of observability will look like

Telemetry volumes will continue to grow. Distributed systems introduce more surface area. AI workloads generate additional signals and amplify data scale. Investigations are becoming more collaborative and more exploratory.

In that environment, the defining characteristic of observability systems will not be the number of features they expose, but the architecture beneath them.

When Slack explodes and dashboards slow down with (or completely stop) answering the right questions, the architecture underneath will determine whether teams find the root cause in minutes or watch the system buckle all over again.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?