The Bit Bucket

The following diagram shows how different messaging technologies — Kafka, Event Hubs, and RabbitMQ — fit into a real-time pipeline.

At the top, we have our event producers: applications generating user activity, IoT sensors sending telemetry, and financial systems pushing transactions. These are the raw sources of events.

Now, those events need to get transported reliably to downstream consumers. That’s where our three options come in:

• Kafka, on the left, is like a freight train. It’s open source, distributed, and designed to move massive volumes of events continuously. It’s ideal for streaming pipelines at very large scale.
• Event Hubs, in the middle, is essentially the managed version of Kafka on Azure. You get the same freight train power, but as a managed service. You don’t worry about the tracks or the engines — Azure handles that for you. It integrates directly with Fabric, which makes it especially attractive if you’re already in the Microsoft ecosystem.
• RabbitMQ, on the right, is like a courier van. It’s not designed to move massive volumes, but it’s incredibly reliable and flexible for routing and guaranteed delivery. If you need every message delivered exactly once — even if the consumer is offline for a bit — RabbitMQ is your best option.

Finally, at the bottom, we have the consumers: dashboards, analytics systems, or applications that depend on this event data. The key is that each broker delivers data in a way that best matches its design philosophy: high throughput (Kafka), managed scale (Event Hubs), or guaranteed precision (RabbitMQ).

2026-02-14

I had a discussion with a friend the other day who wouldn’t believe me that T-SQL allowed an UPDATE operation to be executed against a function in some situations. I must admit that when I first heard about it, it did my head in a bit as well, at least based on all I thought I knew about programming languages in general.

You can only perform an UPDATE against a table-valued function in T-SQL when the function was declared as an inline TVF. It still feels to me like it shouldn’t ever be permitted, but if you want to try it, here’s an example:

2026-02-13

RabbitMQ is another widely used message broker, but it fills a slightly different role compared to Kafka or Event Hubs. It’s an open-source broker that primarily implements the AMQP protocol, though it supports other protocols as well.

Where RabbitMQ shines is in reliable delivery and flexible routing of messages. It’s extremely good at guaranteeing that each message gets to the right destination, even if the consumer isn’t available at the time. You can define queues, exchanges, and routing rules to handle very complex message delivery patterns.

2026-02-12

It was great to catch up with Ben Weissman today and to have him on a SQL Down Under podcast.

Ben is a fellow Data Platform MVP.

Ben focusses on datawarehousing and analytics, still with a lot of on premises work but of course also Fabric / Azure work.

He has long experience with SQL Server, having started with SQL Server 6.5.

Ben says he loves to travel and loves food. He’s an author of books and video courses, and the organizer of data related events like Data Grillen, New Stars of Data, SQL Konferenz, and Data Saturday in both Vienna and Germany and other locations.

2026-02-11

I’m hoping to have Drew Robbins on my new Tech Career for Life Podcast later this week, and I noticed that he had a new book out, so I decided I’d read it before I had him on the show.

In a working world obsessed with metrics — titles, promotions, performance reviews When No One’s Keeping Score lands with a quiet but deliberate challenge: what if the work that matters most is the work that never shows up on a scoreboard?

2026-02-10

One of the discussion lists that I participate in, had a brief discussion the other day about whether or not it’s possible to perform log shipping between differernt versions of SQL Server.

Specifically, can you do log shipping between SQL Server 2017 and SQL Server 2025?

Partial success

SQL Server does support restoring earlier version databases on later versions of the product. The databases get upgraded along the way when you perform restores of databases. SQL Server also allows you to restore transactions logs from earlier versions of the product but the upgrade doesn’t happen until recovery of the database occurs. And that’s why you can’t use STANDBY mode in this situation.

2026-02-09

When we talk about protocols for sending and receiving streams of event data, two of the most common you’ll come across are HTTP and AMQP.

HTTP

HTTP is the workhorse of the web. It’s everywhere, it’s simple, and it’s supported by almost every platform and device. The model is request/response: the client asks for something, the server replies, and then the connection is done. HTTP is stateless, meaning every request is independent. That simplicity makes it easy to use, but it also makes it less suitable for continuous, real-time data flows. If you want to stream updates constantly, you either have to keep making new HTTP requests or hold the connection open in ways that HTTP wasn’t originally designed for.

2026-02-08

A while back, I got an interesting reply to a blog entry about LINQ and Entity Framework terminology. The reader asked if I’d worked with the Progress DBMS.

I have.

Here are my thoughts on it. Obviously others will have a different view of history but I mention Progress over and over again when I’m teaching classes as three examples related to them really hit home for me.

Background

I worked with Progress for many years, starting in about 1983 through to some time in the late 1990’s.

2026-02-07

Azure IoT Hub is a specialized service designed specifically for connecting and managing IoT devices at scale. While Azure Event Hubs is a general-purpose streaming service, IoT Hub focuses on the unique challenges that come with millions of devices out in the real world.

First and foremost, IoT Hub provides secure, reliable connections for those devices. In an IoT deployment, you might have sensors, machines, or even entire fleets of vehicles sending telemetry data up to the cloud. IoT Hub can support millions of devices connecting simultaneously, each streaming their data in real time.

2026-02-06

Crocodiles fascinate me. They’ve been around since the dinosaurs, and until we came along and made weapons, they were a top-tier predator.

Recently I watched the end of a wonderful documentary from the BBC called Dragons Alive . Although my wife didn’t really want to discuss it at 12:30am, it really got me thinking about the nature of curiosity and its importance in how we advance science and through it, society.

2026-02-05