Fabric-Rti-101

Routing is one of the features that makes Eventstreams especially powerful in Fabric. It gives you fine-grained control over where different types of events end up. Instead of building separate ingestion pipelines for every scenario, you can manage multiple flows in one stream and branch the data based on rules you define.

For example, imagine a company that handles both IoT telemetry and financial transactions. IoT data, like sensor readings, is most valuable in a KQL database, where you can run fast time-series analysis and detect anomalies. At the same time, financial transactions are better suited to a warehouse, where structured schemas and BI reporting tools can shine. With routing, both event types can enter the same Eventstream, but the system automatically directs them to the most appropriate destination.

2026-05-03

KQL databases are a specialized storage option in Fabric designed specifically for high-volume event and telemetry data. KQL stands for Kusto Query Language, which comes from the Azure Data Explorer (ADX) engine.

These databases are optimized to handle workloads where you might have billions of small events — like application logs, IoT telemetry, or time-series data — and you need to query them at speed.

Where a warehouse is optimized for structured, relational data and a lakehouse is great for mixing structured and semi-structured data, a KQL database shines when you need to scan and aggregate across massive volumes of events very quickly. You can run queries that look back over millions of log entries or thousands of IoT readings and get sub-second responses. That kind of responsiveness is what makes it possible to power real-time dashboards, alerting systems, and anomaly detection workflows.

2026-05-01

Apache Kafka is one of the most widely used event streaming systems in the world, and for good reason. At its core, Kafka is a distributed, open-source platform that makes it possible to capture, process, and deliver millions of events per second with high reliability.

Kafka organizes data into topics, which you can think of as named channels. A producer writes events into a topic — this could be an application logging user activity, or a payment system recording transactions.

2026-04-29

In most Eventstream transformations, we use the drag-and-drop UI — managing fields, filters, joins, and aggregations visually. But for more advanced scenarios, you can switch to Custom SQL Code mode.

This option allows you to write SQL-style expressions directly against your streaming data.

It’s particularly useful when the visual editor can’t express the full logic you need — for example, if you want to perform complex conditional transformations, apply multi-step calculations, or combine filters and aggregations in a single operation.

2026-04-27

Expanding is the opposite of aggregation. Instead of rolling up many events into a single summary, expansion takes a single complex event and splits it into multiple simpler events. This is really important when one incoming record carries more than one piece of information.

A good example is a log entry that contains multiple error codes in a single message. If we treat that as one event, we lose the ability to analyze each error code individually. By expanding, we break it into separate events — one per error code — which makes it much easier to count, group, and detect patterns across error types.

2026-04-25

Another powerful transformation in real-time processing is the union. A union allows us to take multiple input event streams and merge them into a single, consolidated stream. This is especially useful when we have similar data sources that we want to treat consistently in downstream processing.

A common scenario is application logs. You might have several applications, each producing its own stream of log events. Instead of maintaining separate pipelines for each one, we can use a union to combine them all into a single log stream. From there, we can apply filters, mappings, or aggregations in a single place, which makes management much easier.

2026-04-23

A snapshot transformation is about capturing the current state of something from a continuous stream of events. Instead of storing every single historical record, we keep just the most recent update for each entity.

Think of IoT devices as an example. Each device may send a temperature reading every few seconds. Over time, that becomes a huge stream of data. But often, what the business really wants to know is simple: what’s the latest temperature reading from each device right now? That’s where a snapshot comes in. It continuously updates a table of ‘current values’ as new events arrive, so at any given moment, you can query the most recent state without reprocessing the entire history.

2026-04-21

A hopping window is a flexible windowing strategy that combines elements of both tumbling and sliding. Like tumbling, hopping windows have a fixed length — for example, 10 minutes. But unlike tumbling, they are allowed to overlap, because they advance by a smaller step size than their length.

For example, let’s say we configure a 10-minute window that hops forward every 5 minutes. That means between 12:00 and 12:10, you get one window, and between 12:05 and 12:15, you get another. Each window is 10 minutes long, but because they’re starting 5 minutes apart, they overlap. This means that every event can contribute to more than one window.

2026-04-19

A session window works very differently from tumbling or sliding windows. Instead of cutting streams into fixed blocks of time, a session window groups events based on periods of user or device activity. The window continues to collect events as long as there is activity, and it automatically closes after a defined period of inactivity.

For example, imagine a customer browsing an online store. They click through pages, add items to their cart, maybe watch a video. All of that activity within, say, a 10-minute span of clicks would be grouped into one session. If the customer stops interacting for more than 10 minutes, the window closes, and the next click starts a new session.

2026-04-17

A sliding window is a type of temporal window where the windows are of fixed length, but they overlap because they slide forward by a smaller step than their total size. This makes them perfect for producing smooth, continuous metrics like rolling averages.

For example, suppose we define a sliding window of five minutes that advances every one minute. That means the system calculates results based on the last five minutes of events, then slides forward by one minute and recalculates. Each window overlaps the previous one, so every result shares a lot of the same data, but with just a bit of new information added.

2026-04-15