The Bit Bucket

When working with real-time data, one of the biggest challenges is signal versus noise. Not every event that arrives is valuable for analysis or action. For example, IoT devices may send thousands of telemetry points per second, but only a small fraction of those actually represent unusual or meaningful behavior.

That’s where filtering comes in. Filtering lets us apply simple conditions to events right at the ingestion or processing stage. For instance, imagine we have a stream of temperature readings coming from industrial sensors. Most readings might sit between 20 and 50 degrees Celsius — perfectly normal. But maybe we only care if the temperature goes above 80°C, because that indicates a possible overheating issue. With a filter, we can discard all the normal events and only pass through the ones that require attention.

2026-03-18

Over the years, I’ve seen so many discussions regarding the certification process and exams. I’ve seen posts from many people that are very experienced with products saying they can’t see any point in the certification exams and also argue that competencies in the Microsoft Partner program shouldn’t be based on exams. They feel these people should somehow just be recognised for their other contributions.

Grandfathering

Regarding the certification process, I don’t agree that anyone should be just grandfathered in. Any of the people that have a great deal of knowledge and experience really should be able to just take the exams and be done with it.

2026-03-17

Once events are flowing through an Eventstream, the next decision is: where should they go? This is where outputs come into play.

Fabric supports several key destinations. You can send events into a Lakehouse, which is ideal for combining real-time streams with historical data and keeping a permanent record for later analysis. You can push data into a Warehouse for structured reporting and BI queries. Or you can use a KQL database if your focus is on fast, interactive queries over logs, telemetry, or time-series data.

2026-03-16

Something I really struggle with in this industry is when newcomers to a system want to change standards within existing systems because they think something else is better. It’s a sign of immaturity yet it often applies to people who should be senior. Many system architects fall into this category.

What’s in a PK name?

For example, a vendor system that I’ve been working with has single column primary keys in all tables, and all the primary key columns are named PKey_ID (I’ve changed it a bit to protect the guilty). Now I can’t say I like that naming at all, but that’s not the point. There are a large number of tables that already have that naming scheme.

2026-03-15

When we look at event processing inputs, the first thing to know is that Fabric supports a broad range of streaming sources. The big four are Kafka, Azure Event Hubs, Azure IoT Hub, and any system that speaks AMQP. These cover most of the event-driven architectures you’ll see in the real world, from enterprise message brokers to IoT device fleets and large-scale cloud-native streaming pipelines.

Another important point is that inputs can come from both cloud and on-premises environments. Many organizations are in hybrid mode — perhaps you’ve got a Kafka cluster still running in your datacenter, while also using Event Hubs in Azure for new workloads. Fabric Eventstreams can connect to both, allowing you to bring all those events into a unified pipeline without needing to modernize everything at once.

2026-03-14

When writing and working with T-SQL in SQL Server, you’ll often come across the number 1033. In many cases, that will have the word English associated with it. But what are these numbers, and which ones does SQL Server know about?

The number 1033 comes from the reference to a Windows Locale ID. You’ll find the list of Windows Locale IDs here:

Microsoft Locale ID List

It’s a big list with entries that look like this:

2026-03-13

An Eventstream is Microsoft Fabric’s native service for managing the flow of real-time data. Think of it as a pipeline that sits in the middle: it connects your data sources on one side, applies any transformations you need, and then routes the events to their destinations.

The key thing is that Eventstreams are built into Fabric. You don’t have to spin up infrastructure, manage clusters, or write custom code. Instead, you configure inputs and outputs through the Fabric interface. For example, you might take input from an Azure Event Hub, filter or transform the events, and then send them into a Lakehouse, a KQL database, or straight into Power BI.

2026-03-12

The on-premises versions of SQL Server have the ability to connect one server to another via a mechanism called Linked Servers.

Azure-based SQL Server databases can communicate with each other by a mechanism called External Tables. I’ll write more about External Tables again soon.

Common performance issue with linked servers

With Linked Servers, I often hear people describing performance problems. There’s a configuration setting that commonly causes this, yet it seems to be almost unknown.

2026-03-11

I mentioned in a previous post that the standard storage format for Microsoft Fabric is in OneLake. In fact, you’ll find that pretty much everything that you use in Fabric ends up being stored in OneLake.

The thinking is that you’d be able to look at the data that was captured and being used by one tool then by using a different tool and so on. Now that that all makes sense, it’s not 100% true yet, but the story is getting better.

2026-03-10

I’m not a great fan of linked servers in SQL Server but they are often necessary. If I’m working with the latest version of SQL Server, I really prefer to use External Data Sources and External Tables. But not everyone is on the latest version. In the meantime, what I see all the time, is people hardcoding server names like this:

SDUPROD2022.WWIDB.Payroll.Employees

That makes your code really hard to manage. One option to get around that is to use synonyms.

2026-03-09