Microsoft-Fabric

Many organizations already have huge amounts of business-critical data sitting in relational databases — SQL Server, PostgreSQL, MySQL, and so on. But by default, these databases aren’t real-time data sources. Traditionally, if you wanted to know what changed, you’d run queries at intervals — maybe once an hour, or once a day — and look for differences. That approach works, but it introduces delay.

This is where Change Data Capture, or CDC, comes in. CDC is a technique that turns a traditional database into a real-time data source. Instead of polling for differences, CDC actually streams the changes themselves as they occur. Anytime a row is inserted, updated, or deleted, that operation is captured and emitted as an event.

2026-02-16

The following diagram shows how different messaging technologies — Kafka, Event Hubs, and RabbitMQ — fit into a real-time pipeline.

At the top, we have our event producers: applications generating user activity, IoT sensors sending telemetry, and financial systems pushing transactions. These are the raw sources of events.

Now, those events need to get transported reliably to downstream consumers. That’s where our three options come in:

• Kafka, on the left, is like a freight train. It’s open source, distributed, and designed to move massive volumes of events continuously. It’s ideal for streaming pipelines at very large scale.
• Event Hubs, in the middle, is essentially the managed version of Kafka on Azure. You get the same freight train power, but as a managed service. You don’t worry about the tracks or the engines — Azure handles that for you. It integrates directly with Fabric, which makes it especially attractive if you’re already in the Microsoft ecosystem.
• RabbitMQ, on the right, is like a courier van. It’s not designed to move massive volumes, but it’s incredibly reliable and flexible for routing and guaranteed delivery. If you need every message delivered exactly once — even if the consumer is offline for a bit — RabbitMQ is your best option.

Finally, at the bottom, we have the consumers: dashboards, analytics systems, or applications that depend on this event data. The key is that each broker delivers data in a way that best matches its design philosophy: high throughput (Kafka), managed scale (Event Hubs), or guaranteed precision (RabbitMQ).

2026-02-14

RabbitMQ is another widely used message broker, but it fills a slightly different role compared to Kafka or Event Hubs. It’s an open-source broker that primarily implements the AMQP protocol, though it supports other protocols as well.

Where RabbitMQ shines is in reliable delivery and flexible routing of messages. It’s extremely good at guaranteeing that each message gets to the right destination, even if the consumer isn’t available at the time. You can define queues, exchanges, and routing rules to handle very complex message delivery patterns.

2026-02-12

When we talk about protocols for sending and receiving streams of event data, two of the most common you’ll come across are HTTP and AMQP.

HTTP

HTTP is the workhorse of the web. It’s everywhere, it’s simple, and it’s supported by almost every platform and device. The model is request/response: the client asks for something, the server replies, and then the connection is done. HTTP is stateless, meaning every request is independent. That simplicity makes it easy to use, but it also makes it less suitable for continuous, real-time data flows. If you want to stream updates constantly, you either have to keep making new HTTP requests or hold the connection open in ways that HTTP wasn’t originally designed for.

2026-02-08

Azure IoT Hub is a specialized service designed specifically for connecting and managing IoT devices at scale. While Azure Event Hubs is a general-purpose streaming service, IoT Hub focuses on the unique challenges that come with millions of devices out in the real world.

First and foremost, IoT Hub provides secure, reliable connections for those devices. In an IoT deployment, you might have sensors, machines, or even entire fleets of vehicles sending telemetry data up to the cloud. IoT Hub can support millions of devices connecting simultaneously, each streaming their data in real time.

2026-02-06

Azure Event Hubs is Microsoft’s fully managed event streaming service. If you’re familiar with Apache Kafka, you can think of Event Hubs as Microsoft’s cloud-native equivalent. It’s designed to handle extremely high volumes of events — we’re talking about millions of events per second — all without you having to stand up and manage complex clusters yourself.

One of the most important features of Event Hubs is that it offers Kafka-compatible endpoint. That means if you already have applications, tools, or client libraries that were built to talk to Kafka, in many cases they can connect to Event Hubs with little or no modification. That’s a huge benefit because it reduces friction for teams migrating workloads into Azure or building hybrid architectures.

2026-02-04

Let’s talk about message brokers and event streams — these are the backbone technologies that make real-time systems work at scale.

At the simplest level, a message broker acts as a middleman between the systems producing events and the systems consuming them. Instead of producers and consumers being tightly coupled — where the producer has to know exactly where to send data and the consumer has to be available at the exact right moment — the broker sits in between and handles that communication.

2026-02-02

Most of the systems we’ve worked with historically are request-driven. In this model, a client asks for information and the server provides it. Think about browsing a website: you type in a URL, your browser requests the page, and the server responds with the content. That’s a pull model — the client decides when it wants data. It’s predictable, it’s synchronous, and it’s been the backbone of web applications for decades.

2026-01-31

The Real-Time hub is the single, tenant-wide entry point for working with real-time data in Microsoft Fabric. Every Fabric tenant has exactly one Real-Time hub, and it exists independently of individual workspaces.

Single logical place for streaming data

The Real-Time hub is the single, tenant-wide entry point for working with real-time data in Microsoft Fabric. Every Fabric tenant has exactly one Real-Time hub, and it exists independently of individual workspaces.

2026-01-29

We’ve talked about events, streams, ingestion, and processing — but all of that only really matters if we take the final step: taking action. Actions are what turn insight into outcomes.

It’s one thing to detect that something unusual has happened. Maybe you see a spike in failed logins, a sudden drop in sales, or an overheating sensor. But if you stop at simply noticing it, the value is lost. The real payoff comes when the system — or the people using it — can respond.

2026-01-27