The Bit Bucket

When we talk about KQL databases in Fabric, we’re referring to databases that are specifically optimized for high-volume, time-series, and log-style data. These databases are designed around the kinds of workloads that come from telemetry, sensors, applications, and services that continuously generate events.

Rather than being general-purpose like a relational database, a KQL database is built to handle append-only, event-driven data at scale — often millions of rows per second. The architecture is different from what we see in traditional SQL databases. Data in a KQL database is typically stored in compressed, columnar form, which makes it extremely efficient to query across large time ranges or to aggregate over millions or billions of records.

2026-05-09

I’ve recently been writing about the need for stored procedures to have contracts . Temporary tables add another dimension to that discussion.

Tempory tables are visible within the scope where they are declared but also in sub-scopes. This means that you can declare a temp table in one stored procedure but access it in another stored procedure that is executed from within the first stored procedure.

There are two reasons that people do this. One reason is basically sloppy code, a bit like having all your variables global in a high level programming language. But the more appropriate reason is to avoid the overhead of moving large amounts of data around, and because we only have READONLY table valued parameters.

2026-05-08

Once your Eventstream is processing data, the next key step is deciding where those events should go — their destinations.

Eventstreams are designed to be flexible: you can route events to multiple destinations at once, both inside Fabric and to external systems. Let’s look at the main ones.

First is the Eventhouse, which provides a high-performance, KQL-based analytical store. It’s ideal when you need to query and visualize live data in real time — for example, detecting anomalies or monitoring live operations. Because it uses KQL, it integrates tightly with Real-Time Dashboards and KQL Querysets in Fabric.

2026-05-07

A few days ago, I wrote about SQL CLR and how I don’t normally use it now, but if I did, which types of objects make sense for it. I briefly mentioned user-defined data types but today, I wanted to call out another limitation of these that I’d like to see addressed (if we keep on using SQL CLR).

Early versions of the user-defined data types in SQL CLR had a limitation on size, where they needed to be serializable within 8KB. That limit is now long gone and so the ability to define new data types using SQL CLR integration was now almost at a usable level, apart from one key omission: indexes.

2026-05-06

When it comes to designing an Eventstream pipeline in Fabric, the process generally follows a clear, three-step flow. First, you start with the inputs — the data sources. This might include Kafka, Event Hubs, IoT Hub, or other streaming systems. At this stage, you define the connections and schemas so Fabric knows how to interpret incoming events.

The second step is where you apply transformations. These are the operations that make raw data more usable and more valuable. You might apply filtering to reduce noise and drop irrelevant events. You might use mapping to rename fields, adjust types, or flatten JSON into something cleaner. And you might use routing to branch different types of events to different destinations. Together, these transformations ensure that events are shaped, cleaned, and directed properly before they move downstream.

2026-05-05

It was great to catch up with Jess Pomfret today and to have her on a SQL Down Under podcast.

Jess is a Data Platform Engineer and a dual Microsoft MVP. She started working with SQL Server in 2011, and she says she enjoys the problem-solving aspects of automating processes with PowerShell.

Jess also enjoys contributing to dbatools and dbachecks, two open source PowerShell modules that aid DBAs with automating the management of SQL Server instances.

2026-05-05

I’ve recently been talking to clients about SQL CLR objects. When these were first introduced in SQL Server 2005, many of us had high hopes for them. SQL Server has never been great in regard to extensibility and this provided some way to extend the product.

Nowadays, I avoid SQL CLR. And that’s a real pity. But it’s no longer supported in Azure SQL Database, apart from the system CLR objects of geometry, geography, and hierarchyid. (Note: I’m also not a fan of hierarchyid). I need to use extensibility methods that are available in the different environments that I work in, and Azure SQL Database is one of those. The same applies to Fabric SQL Database.

2026-05-04

Routing is one of the features that makes Eventstreams especially powerful in Fabric. It gives you fine-grained control over where different types of events end up. Instead of building separate ingestion pipelines for every scenario, you can manage multiple flows in one stream and branch the data based on rules you define.

For example, imagine a company that handles both IoT telemetry and financial transactions. IoT data, like sensor readings, is most valuable in a KQL database, where you can run fast time-series analysis and detect anomalies. At the same time, financial transactions are better suited to a warehouse, where structured schemas and BI reporting tools can shine. With routing, both event types can enter the same Eventstream, but the system automatically directs them to the most appropriate destination.

2026-05-03

I was wondering today how often EULAs (end user license agreements) get tested in courts, and in particular, EULAs that appear in consumer-grade applications.

While most sound quite official, it’s hard to imagine most of them being very enforceable. Does anyone EVER read them?

Fair Cop !

I was amused a few years back when I was installing an application, clicked over the EULA and the application said “how could you possibly have read that in 1.076 seconds?”. Yep, got me there; that’s a fair cop.

2026-05-02

KQL databases are a specialized storage option in Fabric designed specifically for high-volume event and telemetry data. KQL stands for Kusto Query Language, which comes from the Azure Data Explorer (ADX) engine.

These databases are optimized to handle workloads where you might have billions of small events — like application logs, IoT telemetry, or time-series data — and you need to query them at speed.

Where a warehouse is optimized for structured, relational data and a lakehouse is great for mixing structured and semi-structured data, a KQL database shines when you need to scan and aggregate across massive volumes of events very quickly. You can run queries that look back over millions of log entries or thousands of IoT readings and get sub-second responses. That kind of responsiveness is what makes it possible to power real-time dashboards, alerting systems, and anomaly detection workflows.

2026-05-01