The Bit Bucket

Google Cloud Pub/Sub is a fully managed event ingestion and messaging system.

It supports large-scale, asynchronous communication between services. It is widely used for event-driven architectures, analytics pipelines, and IoT telemetry.

In this integration model, Pub/Sub acts as a cross-cloud source for Fabric. Messages published to topics in Google Cloud - such as application logs, telemetry events, or transaction data - can be exported or streamed into Fabric Eventstream for analysis.

2026-10-16

AI Ethics is a really timely subject. I recently read AI Ethics in Action: A practical guide to building modern organizations through purposeful, ethical AI innovation as a review copy I received from my friends at PackT.

Author

Laura Miller is described as a continental philosopher and AI ethicist. She works across many continents and is recognized by publications like The New York Times. Laura founded Aeterne and Mindher. I liked seeing that her advisory clients included NIST, the US State Department, and the EU AI Act Public Working Group. Laura is well-positioned to write this book.

2026-10-15

Spark Structured Streaming is part of Apache Spark’s ecosystem and is designed for continuous processing of data streams using a micro-batch model. Unlike true event-by-event streaming engines, Spark processes data in very small, frequent batches - giving it a balance between throughput and fault tolerance.

This approach makes Spark Structured Streaming especially effective for large-scale ETL and data transformation workloads. It can ingest data from a variety of real-time sources, perform complex operations such as joins, windowed aggregations, or enrichment, and then write the results to multiple destinations.

2026-10-14

Version 28 of our free SDU Tools for developers and DBAs is now released and winging their way out to our SDU Insiders.

You can find details on the tools here .

If you haven’t been using SDU Tools yet, I’d suggest downloading them and taking a look. At the very least, it can help when you’re trying to work out how to code something in T-SQL.

This is a huge release of the tools.

2026-10-13

Apache Flink is a distributed stream processing framework designed for real-time analytics and event-driven applications. It’s well known for its ability to handle large volumes of continuous data with stateful computation, meaning it can maintain context across many events - something that simpler streaming systems struggle with.

A key strength of Flink is its support for Complex Event Processing (CEP).

CEP allows you to define and detect patterns of events rather than just single occurrences. For example, you could detect a sequence such as three failed login attempts followed by a successful one, or identify correlated sensor readings that indicate a developing fault.

2026-10-12

There are a few places in T-SQL where instead of a time value, you need to work with a duration.

An example is if I want to sleep for a short period. In T-SQL, we can do that with the WAITFOR statement, used like this:

This command would cause SQL Server to wait for 10 minutes. One of the challenges though, is how do I create this duration string in a programmatic way? For example, how do I create the string ‘00:02:00’ if I’m starting with a value of 120 seconds?

2026-10-11

Kafka Streams is a client-side stream processing library that operates directly on Kafka topics. Unlike a separate processing engine, it runs within your own application or service, allowing data transformation to occur close to where the events are produced.

This makes it ideal for preprocessing data before it reaches Fabric. You can use Kafka Streams to perform tasks such as data enrichment, filtering, or lightweight aggregation. For instance, before sending events to Fabric, you might:

2026-10-10

I have previously posted information about how to get local date and time happening in Azure SQL Database. Many people have been frustrated about the inability to set any date and time to work with, rather than just UTC.

I posted here about calculating it, and here about how to set it for your session.

While the workarounds were helpful, there were many situations where they just weren’t enough. One simple example is where you’re offering a Software as a Service (SaaS) application and want each tenant database to have its own date and time configuration, but many other people just wanted their database to reflect their own local time.

2026-10-09

An Eventhouse-originally designed for realtime event data-can also serve as a vector database in Fabric RTI, supporting semantic/embedding-based retrieval workflows.

The key enabler is the dynamic column type, which allows you to store unstructured data such as arrays or JSON blobs. This lets your embedding vectors live alongside your ordinary event metadata.

Microsoft recommends using Vector16 encoding (i.e. 16-bit floating representation) rather than full 64-bit. This dramatically reduces storage footprint and boosts the performance of vector operations like dot product or cosine similarity.

2026-10-08

AI development is another really timely subject. I recently read Hands-On AI Development with Python: Build and Deploy Real-World AI, Machine Learning, Deep Learning, and NLP Applications as a review copy I received from my friends at PackT.

Author

Vivian Aranha is described as an AI educator, technology leader, and founder of School of AI. He was worked with Fortune 500 organizations including The Washington Post, Delta Air Lines, and IBM. He currently teaches globally through School of AI, Udemy, Skool, and Maven.

2026-10-07