What if a company could detect a failure, anticipate a purchase, or block fraud while the event was still happening? That is the proposal from IBM and Confluent: combine foundation models for time series with enterprise data in streaming.
The collaboration brings IBM Granite Time Series models directly to Confluent Cloud, where they can run inside Apache Flink. The goal is to reduce the distance between a changing data point and the decision that depends on it.
From Isolated Models to Real-Time Intelligence
Until now, working with time series has usually meant building a specific model for every product, sensor, card, or production line. That process requires prepared data, feature engineering, specialists, and months of work.
That is why many companies model only their most important time series. The rest are covered with additional inventory, reserve capacity, or safety margins. It is an understandable strategy, but an expensive one: by the time the prediction arrives, the opportunity to act may already be gone.
Foundation models for time series, known as TSFM, aim to change that dynamic. They are trained on varied signals and can generalize to time series they have never seen before. From a window of data, they can estimate what will happen next, detect unusual behavior, find similar historical episodes, and evaluate which configuration could reach a target.
The idea is not for every team to become an artificial intelligence lab. The idea is to turn complex capabilities into functions that a business expert can use.
What the Confluent Integration Brings
Confluent keeps enterprise data moving. Industrial sensors, payments, application metrics, and operational events can continuously arrive in its data streams.
IBM Granite Time Series uses that updated context to generate forecasts, detect anomalies, fill in missing data, classify behaviors, find similar episodes, and support optimization processes.
Inference runs natively inside Confluent Cloud for Apache Flink. This avoids extracting time series to another machine learning platform or data warehouse before analyzing them.
The integration also allows Flink to manage the state of each time series. This matters because a prediction depends on recent history, and an anomaly only makes sense compared with a normal pattern that is updated over time.
The announced benefits include:
- Intelligence where the data lives: models work directly on real-time streams.
- Simplified configuration: Confluent manages the model service, infrastructure, scaling, and environment operations.
- Up-to-date context: results are based on the current state of the business, not on old batches.
- Integrated governance: schemas, lineage, access controls, and traceability remain within the platform.
- Reusable results: predictions can be written to Kafka topics and consumed by alerts, dashboards, lakehouses, or AI agents.
- Less operational complexity: there is no need to maintain dedicated inference servers or GPUs for these use cases.
- Enterprise security: data stays within Confluent Cloud and follows access control policies.
For teams, the practical change is significant. Instead of building an entire architecture around each model, they can call these capabilities through familiar SQL functions.
Four Models for Different Questions
IBM and Confluent are not presenting a single model for every scenario. The proposal includes four foundation models for time series, available in Early Access and usable through Flink SQL’s AI_FORECAST and AI_DETECT_ANOMALIES functions.
The choice depends on specific questions: will you analyze a single time series or hundreds of thousands? Are there several related variables? Do you need a distribution of possible outcomes? Is the priority forecasting or anomaly detection?
PatchTST-FM
PatchTST-FM processes signals in time-based fragments, somewhat like how a language model processes parts of a text. Each variable keeps its own channel, reducing the risk that a noisy signal will distort the others.
It can also provide a distribution of outcomes rather than just a single number. This allows a planner to choose, for example, an inventory level associated with the 90th percentile of expected demand.
FlowState
FlowState maintains an updated summary with every new data point. Its design is intended for continuous behavior over time, from SCADA signals captured every few seconds to hourly financial data.
TTM
TTM uses small networks to combine temporal and multivariate information. Its compact size allows inference to run on CPUs and cover large numbers of time series—even hundreds of thousands of them—without relying on expensive infrastructure.
TSPulse
TSPulse combines representations from the temporal and frequency domains. It is designed for tasks such as anomaly detection, classification, data imputation, and finding similar behaviors.
The models can be switched with an SQL parameter, without redesigning the entire data stream. An example forecast is:
SELECT
AI_FORECAST(
load_kw,
event_time,
JSON_OBJECT('model' VALUE 'ttm', 'horizon' VALUE 12)
) OVER (
ORDER BY event_time
RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW
) AS forecast
FROM meter_readings;
The value of model can be changed to test another model in the portfolio. The query and the overall pipeline remain intact.
Use Cases: Inventory, Fraud, and Factories
Imagine a supermarket chain that needs to forecast demand across its entire catalog. Foundation models can work with products they have never seen before, incorporate factors such as promotions or weather, and produce a distribution of outcomes.
That makes it possible to turn a forecast into a policy: how much to replenish to reach a given service level, when to allocate inventory to a store, or when to apply a discount before a product loses value.
In this scenario, each result can trigger an action. The forecast arrives in a Kafka topic and is consumed by replenishment, pricing, allocation, and planning systems. It is no longer just about generating a report, but about feeding decisions while the outcome can still change.
Anomaly detection offers another example. At a bank, the system can maintain a normal pattern for each card and evaluate every payment while it is still taking place.
A routine purchase can go through without friction, while a transaction that suddenly departs from historical behavior can trigger a review. The model can also search for similar cases, so the analyst receives not just an alert but context about what happened in previous situations.
In an industrial plant, data on temperature, speed, pressure, viscosity, and output can feed a model in real time. The system could anticipate a deviation, simulate how the result would change with other parameters, and recommend a configuration that respects process constraints.
The difference is that the plant expert can define the objective—such as increasing throughput or reducing energy consumption—without having to build a machine learning model from scratch.
The Value of Finding Precedents
One especially interesting capability is similarity search. The models turn a behavior window into a vector, a compact representation that makes it possible to compare similar shapes even when they have different scales or values.
This allows a company to answer questions such as:
- Which products had similar demand during their launch?
- Which previous runs behaved like this production line?
- Which fraud cases resemble this transaction?
- What happened the last time a piece of equipment showed this pattern?
The result is not just a score. It is a precedent that can give an investigation, an AI agent, or a human decision more context.
Availability and Next Steps
The integration is in Early Access on Confluent Cloud for AWS. The initial proposal includes forecasting and anomaly detection directly on data streams, without requiring additional training, feature engineering, or specialized AI expertise.
Confluent Platform will be the next step, with the goal of bringing the same models to on-premises and hybrid environments. IBM also highlights model provenance, license transparency, and its enterprise governance framework.
The strategy is clear: move from isolated models that generate reports to capabilities integrated into the systems where decisions happen. When a sensor deviates, a customer makes a payment, or demand changes, waiting for the next data batch may be too late.
Time series intelligence becomes more valuable when it not only explains what happened but also helps you act while the event is still unfolding. That is the real contribution of this integration: turning live signals into forecasts, alerts, and actions that can enter the operational flow directly.
Original Source
https://huggingface.co/blog/ibm-research/real-time-intelligence