The Bit Bucket

In KQL, the union operator can combine results from multiple tables, even when those tables follow a naming pattern. This is particularly useful in environments where data is split across many similar tables — for example, logs_2024_01, logs_2024_02, and so on.

You can use wildcards with the union operator to simplify these queries. Instead of listing each table individually, you can write:

union withsource=TableName logs_*

This automatically brings together all tables whose names begin with logs_.

2026-09-16

When designing databases, one question that comes up all the time is how large columns should be.

Numbers

For numbers, the answer is always big enough but not too big. This week I’ve been working at a site where the client numbers were stored in int columns. Given the clients are Australians and the Australian Bureau of Statistics Population Clock says there are just under 25 million of us, an integer seems a pretty safe bet, given it can store positive numbers up over two billion. It’s hard to imagine that number being exceeded, but I’ve seen people deciding that it needs to be a bigint. I doubt that. Even if we count all our furry friends, we aren’t going to get to that requirement.

2026-09-15

In Fabric, KQL supports querying across multiple databases or even across different clusters.

This feature is especially valuable when data is distributed across teams, regions, or functional domains — for example, separate clusters for telemetry, transactions, and logs.

A cross-database query allows you to access tables in more than one KQL database within the same cluster. This is often used when analytical data and operational data are stored separately but need to be correlated — such as linking user activity logs with transaction histories.

2026-09-14

Time to read

I was on a flight from Houston to Washington via Dallas a while back. We seemed to sit forever on the tarmac in Dallas, in fact just over 1.5 hours from leaving gate to taking off. It was amazingly silly. We were 18th in the takeoff queue and the airport required the planes to keep turning their engines off after moving each time. The captain (who was clearly fed up) warned us that if he had to stop and restart the engines one more time, we’d have had to return to the terminal for more fuel. And we hadn’t even left Houston.

2026-09-13

Partitioning is one of the most effective ways to improve scalability and performance in event-driven systems. It works by dividing a single event stream into multiple independent subsets, or partitions, that can be processed in parallel.

Each partition operates independently, which allows multiple consumers or processing nodes to work simultaneously without interfering with one another. This parallelism increases throughput and helps maintain low latency even as data volumes grow.

The partition key determines how events are distributed among partitions. Common choices include device ID, user ID, or region, depending on the data model and access patterns. The key should be selected carefully:

2026-09-12

If you are aware of Betteridge’s law of headlines , you already know the answer, but let me explain.

There are a lot of odd myths that surround SQL Server. One of the more persistent ones is related to indexes on columns that hold bit values.

A SQL Server column that’s a bit can be in three states. It can have the value zero; it can have the value one; and it can be NULL (ie: have no value at all).

2026-09-11

In Kusto Query Language (KQL), materialized views are a key optimization feature that improve the performance of repeated or computationally expensive queries on streaming data. Normally, when you query a real-time data stream, the system must process the incoming events and apply all calculations each time the query runs. As data volumes increase, this can become inefficient and introduce latency.

A materialized view solves this by precomputing and storing the results of a defined query in a physical table. This table is automatically updated as new events arrive.

2026-09-10

Was 2018 a leap year?

What about the year 2000?

Will 2100 be a leap year?

When I was a young student at school, we learned that leap years were every four years. Then as I got older, I learned that there was more to it than that.

It’s important to understand that any calendar is just an approximation, and there is a lot more to calendars than most people realize.

2026-09-09

Scaling an Eventstream involves more than simply increasing capacity — it’s about ensuring that each stage of the data pipeline can handle higher volumes efficiently and reliably.

The first approach to scaling is by adding more inputs and outputs. As the number of data sources or destinations grows, Eventstream can be configured to handle multiple concurrent data feeds, such as several IoT hubs, Kafka topics, or application logs.

You can also tune the Eventstream’s capacity to handle greater throughput. This involves adjusting the compute and memory resources allocated to the Eventstream so that it can process more events per second without increasing latency.

2026-09-08

There’s a UK game show that I like to watch. It’s called Lingo and it’s the standard word game turned into a full TV show. It’s based on a US game show of the same name.

While watching it, I realized that I’ve come to form an immediate opinion on the education level of the contestants, not based on their appearance, not based on their accents, not based on their assortment of tattoos, but on how they pronounce the letter H.

2026-09-07