The workflow is straightforward once you know the moving parts. You begin in your Eventhouse - that’s where data from your event streams is stored.
First, ensure that the Python plugin is enabled. This plugin provides the AI runtime used to train and run anomaly models. It can take up to an hour to enable for the first time, so plan that in advance.
2026-09-26
This new capability in Microsoft Fabric Real-Time Intelligence brings true AI-driven anomaly detection into the streaming environment. Traditionally, anomaly detection required a fair amount of data-science work - building models, testing parameters, and maintaining code. Fabric now handles much of that automatically.
The service examines incoming time-series or streaming data to look for unusual patterns - things that differ significantly from what’s been observed before. It supports both univariate detection, where you track a single metric such as temperature, latency, or transaction volume, and multivariate detection, which handles correlated signals - for example, temperature, pressure, and vibration from the same machine.
2026-09-24
Machine learning plays an increasingly important role in real-time analytics. Instead of only using models in batch workflows, many systems now apply them directly to streaming data - allowing predictions, classifications, or anomaly detection as events occur.
In this model, each incoming event - such as a transaction, sensor reading, or social media post - can be scored in-flight by a machine learning model. For example, a transaction can be classified as potentially fraudulent, an IoT signal can be checked for signs of equipment failure, or a social post can be analyzed for sentiment.
2026-09-22
Update policies in KQL (and thus in Fabric) are a mechanism to automatically transform ingested data as it arrives.
Rather than separately ingesting raw data and then running background jobs to clean, route, or enrich it, an update policy allows you to declare a transformation query that runs whenever new data lands in a source table.
The transformation query writes its output into a target table. The source and target must exist in the same database. The schema (column types and order) of the transformation output must match what the target table expects.
2026-09-20
KQL offers a wide range of specialized functions designed to handle advanced analytical tasks — particularly useful in real-time and large-scale telemetry environments.
Some of the most important belong to three broad groups:
2026-09-18
In KQL, the union operator can combine results from multiple tables, even when those tables follow a naming pattern. This is particularly useful in environments where data is split across many similar tables — for example, logs_2024_01, logs_2024_02, and so on.
You can use wildcards with the union operator to simplify these queries. Instead of listing each table individually, you can write:
union withsource=TableName logs_*
This automatically brings together all tables whose names begin with logs_.
2026-09-16
In Fabric, KQL supports querying across multiple databases or even across different clusters.
This feature is especially valuable when data is distributed across teams, regions, or functional domains — for example, separate clusters for telemetry, transactions, and logs.
A cross-database query allows you to access tables in more than one KQL database within the same cluster. This is often used when analytical data and operational data are stored separately but need to be correlated — such as linking user activity logs with transaction histories.
2026-09-14
Partitioning is one of the most effective ways to improve scalability and performance in event-driven systems. It works by dividing a single event stream into multiple independent subsets, or partitions, that can be processed in parallel.
Each partition operates independently, which allows multiple consumers or processing nodes to work simultaneously without interfering with one another. This parallelism increases throughput and helps maintain low latency even as data volumes grow.
The partition key determines how events are distributed among partitions. Common choices include device ID, user ID, or region, depending on the data model and access patterns. The key should be selected carefully:
2026-09-12
In Kusto Query Language (KQL), materialized views are a key optimization feature that improve the performance of repeated or computationally expensive queries on streaming data. Normally, when you query a real-time data stream, the system must process the incoming events and apply all calculations each time the query runs. As data volumes increase, this can become inefficient and introduce latency.
A materialized view solves this by precomputing and storing the results of a defined query in a physical table. This table is automatically updated as new events arrive.
2026-09-10
Scaling an Eventstream involves more than simply increasing capacity — it’s about ensuring that each stage of the data pipeline can handle higher volumes efficiently and reliably.
The first approach to scaling is by adding more inputs and outputs. As the number of data sources or destinations grows, Eventstream can be configured to handle multiple concurrent data feeds, such as several IoT hubs, Kafka topics, or application logs.
You can also tune the Eventstream’s capacity to handle greater throughput. This involves adjusting the compute and memory resources allocated to the Eventstream so that it can process more events per second without increasing latency.
2026-09-08