The Bit Bucket

I spend much of my time nowadays working with Microsoft Fabric and there are many opinions on how to use it for the best outcomes, so I was interested to see a new book Analytics Engineering with Microsoft Fabric and PowerBI from friends of mine (in the Fabric community), Shabnam Watson and Nikola Ilic. Nikola sent me a copy to review.

Authors

Shabnam Watson is a business intelligence consultant and owner of ABI Cube, a company that specializes in delivering data solutions using the Microsoft Data Platform. She has decades of experience and is a fellow Microsoft Data Platform MVP.

2026-10-27

A digital twin is a virtual representation of a physical system or asset - such as an engine, a building, a production line, or a logistics network.

Unlike a static model, a digital twin is continuously synchronized with its real-world counterpart through real-time telemetry and sensor data. This constant data flow ensures that the virtual model reflects the current operating state of the physical system.

The purpose of a digital twin extends beyond visualization. It enables simulation, monitoring, and predictive analysis. For example, engineers can test what-if scenarios - such as how a machine might behave under higher load or different environmental conditions - without interrupting operations. It also supports predictive maintenance, by identifying early indicators of wear or failure before they result in downtime.

2026-10-26

A while back, I wrote about the ANSI_NULLS option for tables and after that I had several more questions related to NULL values.

Note that I mentioned NULL values, not values of NULL, and not values equal to NULL. That’s because being NULL is a state that a value is in, not itself a value. That’s why our queries say IS NULL and not = NULL.

Magic values

Now because of that, many developers see values that are NULL as a pain in the neck as they have to be dealt with separately, unlike other values. This can lead to many poor design decisions but I want to start today with the decision to use magic values.

2026-10-25

Several important trends are shaping the future of real-time data systems, moving beyond simple event streaming toward intelligent, autonomous, and action-oriented pipelines.

One key direction is the integration of streaming with AI and machine learning. Instead of passively analyzing data, systems can now make inferences or decisions as events occur - for example, predicting failures, recommending actions, or adapting system behavior in real time. This shift from descriptive to predictive and prescriptive analytics marks an important evolution.

2026-10-24

Today, I was editing some SQL code using the SQL editor in a Microsoft Fabric Warehouse.

I needed to reorder a list of columns.

I find that a pretty frustrating editor but usually I can avoid it by connecting with SSMS. But for some reason, SSMS decided it didn’t want to connect to the warehouse today either.

No problems, I thought I’ll just edit the code with a combination of SSMS, and Excel (to reorder the column names), and then paste the code back into the Fabric SQL editor. But every time I did that, I ended up with a bunch of strange orange boxes all throughout the code.

2026-10-23

Both Fabric Real-Time Intelligence (RTI) and Azure Stream Analytics (ASA) are designed to process streaming data, but they serve different roles within the Microsoft data ecosystem.

Fabric RTI is part of the broader Microsoft Fabric platform. It uses KQL (Kusto Query Language) as its foundation and is tightly integrated with OneLake, Power BI, and other Fabric components such as Eventstream and Activator. This integration makes it suitable for end-to-end real-time analytics, where streaming data is combined with historical data, transformed, visualized, and acted upon - all within one environment.

2026-10-22

I’ve been continuing with my review of famous books. This time, I listed to the AudioBook of The Old Man and the Sea by Ernest Hemmingway. It’s a very long time since I’d listened to this one, and I had forgotten quite a bit of it.

Author

According to Wikipedia, Ernest Miller Hemingway was an American novelist, short-story writer, and journalist who is known for having an economical, understated style that influenced later 20th-century writers. They also note that he has been romanticized for his adventurous lifestyle and outspoken, blunt public image. I can deal with that.

2026-10-21

A KQL Follower Database provides a way to share data across clusters or organizations while maintaining control and performance. Instead of duplicating ingestion pipelines or exporting data manually, a follower database automatically mirrors data from a leader database hosted in another KQL cluster.

This pattern is especially useful when multiple teams, regions, or tenants need access to the same data for analytics.

The follower maintains a read-only, synchronized view of the source. Updates to the leader are automatically reflected in the follower without any manual refresh or re-ingestion.

2026-10-20

Over the years, I’ve had a surprising number of questions on how to output all the data in a table as a series of INSERT statements. SQL Server Management Studio has had the ability to do this for a long time. Here’s an example.

In Object Explorer, I’ve expanded the WideWorldImporters database, then expanded Tables. Where people come unstuck is they right-click the table, and look at the scripting options:

2026-10-19

MQTT is a lightweight messaging protocol designed for IoT and edge environments.

It uses a publish/subscribe model, making it efficient for devices that send small amounts of data over unreliable or bandwidth-limited connections - such as sensors, embedded devices, or mobile units.

In MQTT, a central broker (like Eclipse Mosquitto, HiveMQ, or EMQX) handles communication between clients. Devices publish messages to topics, and subscribers receive updates for the topics they’re interested in. This model decouples producers and consumers, improving scalability and resilience.

2026-10-18