Microsoft-Fabric

Batch processing has been the backbone of data analytics for decades. The idea is simple: you collect data over a period of time, maybe hours or a whole day, and then process it in one big chunk. This is how traditional ETL pipelines and overnight data warehouse loads work. It’s efficient when immediacy doesn’t matter — for example, producing a daily sales report each morning.

But the limitation is obvious: if you need to react quickly, batch just doesn’t cut it. By definition, you’re waiting for the batch window to complete before you see the results. If fraud is happening right now, or if a customer is struggling with your app this very minute, a batch report tomorrow morning is far too late.

2026-01-15

When people hear the phrase real-time, their minds often jump straight to sub-second response. They imagine dashboards updating instantly or transactions being analyzed faster than the blink of an eye. But in reality, not every scenario demands that level of speed — and chasing sub-second performance everywhere can actually be counterproductive.

It’s helpful to think about real-time in classes of latency. At the extreme, you have sub-second responses — critical for areas like algorithmic trading, where even a few milliseconds can cost millions. Then there are scenarios where a response in a few seconds is perfectly adequate — for example, fraud detection. If you can flag a suspicious credit card transaction within two or three seconds, that’s usually enough to stop the transaction before it clears. And then there are other cases where even minutes are fine. Think about a customer experience dashboard in a call center — if the screen refreshes every minute or two, that still feels live enough to be useful.

2026-01-13

When we talk about real-time, it’s worth asking: why does it matter so much? The answer is that in many situations, timing is everything. Traditional business reports are incredibly valuable, but they often arrive hours or days after the fact.

By the time you’ve spotted the issue, the opportunity to act has already passed.

Take fraud detection as an example. If a customer’s credit card is being misused, you can’t afford to wait until the next daily reconciliation. By then, dozens of fraudulent purchases might have gone through. Real-time intelligence means the system can detect unusual patterns on the spot and stop the fraud before it escalates.

2026-01-11

This is the start of a new series of blog posts dedicated to Microsoft Fabric Real-Time Intelligence or RTI. I think RTI is an important aspect of Microsoft Fabric, yet it is currently underutilized. I hope to try to help change that.

So why, RTI ?

Let’s start with a simple idea. Real time intelligence (or RTI) is about shrinking the delay between when data is created and when you can act on it. In traditional systems, we’re often used to data being collected, stored, and only analyzed later, maybe overnight or even weekly. That’s fine for long term reporting, but it’s too slow for situations where immediate action matters.

2026-01-09

More Microsoft Fabric love !

Turn data into insights the moment it happens because yesterday’s data is already too late !

We’ve added more Microsoft Fabric love to our courses. We’re pleased to announce that our new course Mastering Microsoft Fabric Real-Time Intelligence is now available. This is another course that we’ve had so many requests for. You can find details of it here:

Mastering Microsoft Fabric Real-Time Intelligence

Do you need to master Real-Time Intelligence in Microsoft Fabric?

2025-12-14

One of the most important tables in any business intelligence style analytic system is a Date table.

In most formal documentation, that’s ironically referred to as a time dimension, but in so many systems, time isn’t part of it. That’s because the exact time that something happened often isn’t of interest for the aggregated data that often forms the majority of analytic analysis.

Storing time?

Do we ever store the time though?

2025-12-13

Just a heads-up that v26 of SDU Tools went out to SDU Insiders last week. If you haven’t used SDU Tools, they are just a large library of functions implemented in T-SQL. You can use them as a full library or use them as examples of code in work you are trying to do.

Fabric SQL Database Release

The main reason for this release (apart from the standard list updates for SQL Server versions, etc.) is to introduce a new target for SDU Tools. When you download v26, you’ll notice there are now three installation scripts, and three removal scripts. So, there are now versions for:

2025-09-30

Learn to design, automate, and optimize data pipelines across Azure and Microsoft Fabric environments !

We’re pleased to announce that our new course Azure Data Factory and Fabric Data Factory: Data Integration for Data Professionals is now available. This is another course that we’ve had so many requests for. You can find details of it here:

Azure Data Factory and Fabric Data Factory: Data Integration for Data Professionals

Are you ready to master modern data integration with Azure and Fabric Data Factory?

2025-08-11

I was pleased to recently get to review a new book by Nagaraj Venkatesan called Architecting Power BI Solutions in Microsoft Fabric . Nagaraj was an MVP for two years and now a cloud solution architect working for Microsoft.

NOTE: I reviewed the book while it was in preview.

I liked the way that Power BI was discussed for different personas: Business Users, Data Scientists, Administrators.

I was puzzled to see a requirement in the front matter for Angular 9, TypeScript 3.7, and ECMAScript 11. I couldn’t find any relevance to them in the book. Hopefully it will be removed before publication.

2025-04-16

Another Fabric Down Under podcast is out the door.

This time, the guest was Minni Walia.

Minni is a Principal Program Manager with the Microsoft Fabric Customer Advisory Team (CAT), specializing in the Real-Time Intelligence capabilities within Fabric.

Minni has over 20 years in tech, and has extensive experience in developing applications and data platforms. She spent more than a decade as an Enterprise Architect before transitioning into data analytics and engineering roles at Microsoft.

2025-02-10