Let’s look at how a basic KQL query is structured.

Every KQL query starts with a table — that’s your starting dataset, like Events, Logs, or Telemetry. From there, you build a pipeline of operations using the pipe (|) symbol, which passes the output of one operation into the next.

The most common first step is to filter the data using the where operator. It works just like a SQL WHERE clause but uses a more natural, functional syntax. For example:

Events
| where Level == "Error"

That filters your table down to only the rows you care about.

Next, you can use project to select specific columns — similar to SELECT in SQL, but again, it feels more pipeline-oriented:

| project Timestamp, UserId, Message

That step just keeps the columns you want to work with.

Finally, you can use take to limit how many rows are returned — for example:

| take 10

That’s especially helpful when you’re exploring large datasets and just want to see a quick sample of the output.

What’s nice about KQL is that it’s composable — you can chain together as many steps as you like, and each one is easy to read. You can almost think of it like reading a sentence: start with this table, filter it, pick these columns, then take a few rows. That pipeline model is what makes KQL so approachable — it’s simple, expressive, and great for building up queries incrementally.

Learn more about Fabric RTI

If you really want to learn about RTI right now, we have an online on-demand course that you can enrol in, right now. You’ll find it at Mastering Microsoft Fabric Real-Time Intelligence

I had a lot of good feedback on my recent post about how to query group membership for a given login.

One tricky additional question was about how you could let a specfic user be able to find the group membership for another login, without the user being a sysadmin to run the code.

Now doing that is a bit trickier but can be done by creating a certificate, a login from the certificate, then assigning permissions to that login, and finally applying a digital signature to the procedure using the certificate.

Here’s a walkthrough:

Let’s start by creating a certificate that we’ll use for this purpose:

USE master;
GO

CREATE CERTIFICATE WindowsGroupAccessCertificate 
  ENCRYPTION BY PASSWORD = 'VerySecretStuff'
  WITH SUBJECT = 'Windows Group Lookup Access Certificate',
       EXPIRY_DATE = '99990101';
GO

Note: Make sure you use an appropriate password.

Next, we create a login from that certificate. Note that this isn’t a login that can actually log in, but is used as a container for the required permissions:

CREATE LOGIN WindowsGroupAccessLogin
FROM CERTIFICATE WindowsGroupAccessCertificate ;
GO

We need to then add that special new login to the sysadmin role so they can IMPERSONATE any login:

ALTER SERVER ROLE sysadmin
ADD MEMBER WindowsGroupAccessLogin;
GO

Then we create the stored procedure using the same techniques as I mentioned last time. It would be much simpler if we could just use WITH EXECUTE AS in the procedure to temporarily become an administrator but that doesn’t work.

CREATE OR ALTER PROCEDURE dbo.GetWindowsGroupsForLogin
@LoginName sysname
AS
BEGIN
    EXECUTE AS Login = @LoginName;

    SELECT principal_id AS PrincipalID,
           [sid] AS SecurityID,
           [name] AS GroupName,
           usage AS GroupUsage
    FROM sys.login_token 
    WHERE [type] = N'WINDOWS GROUP';
END;
GO

We then digitally sign the stored procedure using the certificate. This will give anyone that runs the stored procedure sysadmin permission but only while they are running the procedure, and only if the procedure has not been modified in any way. Any change to the procedure will cause the digital signature to be dropped:

ADD SIGNATURE TO dbo.GetWindowsGroupsForLogin
BY CERTIFICATE WindowsGroupAccessCertificate
WITH PASSWORD = 'VerySecretStuff';
GO

Just to check that the procedure works ok, I’ll run it as myself first:

That’s all good so let’s now work out if another user can run it:

Hope that helps someone!

It’s time to look at what makes KQL different from traditional query languages.

At its foundation, KQL works with tables, rows, and columns, just like SQL. The structure feels familiar, but the query model is quite different.

KQL queries are built from operators — like where, summarize, project, and extend — and these are chained together using the pipe (|) symbol. Each operator takes the output of the previous one and transforms it further. It’s like building a data pipeline, one step at a time. This makes it incredibly readable and modular — you can easily add, remove, or rearrange steps without rewriting the whole query.

Another key idea is schema-on-read. Unlike traditional databases that enforce a strict schema before data can be queried, KQL can interpret the schema dynamically as the data is read. That makes it very flexible when working with semi-structured or nested data, such as JSON or dynamic fields.

Performance-wise, KQL is optimized for fast scanning of massive datasets. It can process billions of rows in seconds by leveraging distributed storage and indexing under the hood.

Because it’s often used for log and telemetry analysis, time plays a central role. Many queries filter, aggregate, or visualize data by timestamp, which makes it perfect for trend analysis, anomaly detection, or monitoring real-time systems.

Finally, it’s worth noting that KQL is case-sensitive, except for its reserved keywords. That means EventName and eventname could be two different fields, so consistency in naming is important when you write or share queries.

Altogether, these concepts — pipelines, schema-on-read, and time-based analysis — are what make KQL both powerful and well-suited to the world of real-time intelligence.

Learn more about Fabric RTI

If you really want to learn about RTI right now, we have an online on-demand course that you can enrol in, right now. You’ll find it at Mastering Microsoft Fabric Real-Time Intelligence

BACKUP TO URL was introduced as an add-on in Cumulative Update 2 for SQL Server 2012 Service Pack 1 and as a built-in feature for SQL Server 2014. I’ve talked about it in previous blog posts.

We have been using this in a variety of ways from on-premises systems:

For example, it is an easy way to distribute a backup of a database to a large number of systems. Imagine you have a chain of retail stores that needs product and other reference information updated regularly. You can keep this data in a separate database at the head office, back it up to an Azure Storage account, and have each store download it separately.  This has major bandwidth and reliability improvements over other solutions such as having each store maintain a VPN connection to the head office.

As another example, we have clients who simply aren’t able to gain enough space on their SANs to keep enough local copies of their backups.

The more common scenario though, is to use it for backups from Azure Virtual Machines that are running SQL Server. Rather than performing a backup to a virtual machine virtual disk, we often achieve better performance by bypassing the file-system on the virtual disk and backing up directly to a URL. Both end up in Azure storage but backing up directly to storage gives us both better performance, and can help to avoid the drive limit and size restrictions for virtual machines.

Regardless of why you are using BACKUP TO URL, one of the problems that you are likely to run into at some point is the dreaded:

Msg 3271, Level 16, State 1, Line 60
A nonrecoverable I/O error occurred on file
“ https://somestorageaccount.blob.core.windows.net/backups/somedatabase.bak : "
Backup to URL received an exception from the remote endpoint. Exception Message: The remote server returned an error: (412) There is currently a lease on the blob and no lease ID was specified in the request…
Msg 3013, Level 16, State 1, Line 60
BACKUP DATABASE is terminating abnormally.

Applications using Azure storage can take leases on files that are held in the storage containers. This avoids issues with other applications concurrently changing (or even deleting) files that the application needs. The BACKUP TO URL feature in SQL Server takes an infinite lease on the backup file that it creates. That lease is removed when the backup completes.

You can find out more about BLOB leases here and here .

However, if you interrupt a backup (or network issues interrupt it for you), and this is a prolonged interruption, the lease can remain and when you try to overwrite that backup blob (or even delete it), you’ll see the error above.

Now, the design feature that makes this a bit easier to deal with is that the BACKUP TO URL command always uses a well-known lease ID: BAC2BAC2BAC2BAC2BAC2BAC2BAC2BAC2

What is needed to delete it then, is a tool that can break leases, or to run a PowerShell script. You should also take this as yet another hint to learn about PowerShell if you haven’t done so already.

An example of the PowerShell code is as follows:

az storage blob lease break --blob-name "YourBlobName" --container-name "YourContainerName" --account-name "YourAccountName"

KQL, or Kusto Query Language, is a read-only, declarative query language.

Unlike SQL, which is procedural and typically runs in relational systems, KQL is pipeline-based. Each operation feeds directly into the next using the pipe (|) operator. This makes it incredibly intuitive once you get used to it — you start with a dataset, apply filters, transformations, and aggregations step by step, building your query like a flow of operations.

For example, you might start with a table of events, then filter for errors, group them by region, and calculate counts — all in a single, readable flow.

It was originally created for Azure Data Explorer — the system Microsoft designed to handle massive telemetry workloads.

It’s built specifically for large-scale data exploration, particularly for logs, telemetry, and time-series data. Instead of focusing on transactions or updates, KQL is all about quickly slicing, filtering, and analyzing high-volume data. You can think of it as a language designed for insights, not for modifying data.

You’ll find KQL everywhere across Microsoft’s ecosystem: Fabric Real-Time Intelligence, Azure Monitor, Log Analytics, Microsoft Sentinel and Azure Data Explorer (or ADX) all rely on it. So, once you learn it here, you can reuse the same knowledge across a wide range of monitoring and analytics platforms.

While looking into KQL, I recommend you also check out the Kusto Detective Agency. It gamifies learning KQL. You’ll find it at https://detective.kusto.io .

Learn more about Fabric RTI

If you really want to learn about RTI right now, we have an online on-demand course that you can enrol in, right now. You’ll find it at Mastering Microsoft Fabric Real-Time Intelligence

I saw a question on a SQL Server mailing list about how to determine the Windows groups for a given SQL Server login.

That’s actually easy for a sysadmin login, as they have IMPERSONATE permission for other logins/users.

Here is an example procedure:

USE tempdb;
GO

CREATE OR ALTER PROCEDURE dbo.GetGroupsForALogin
@LoginName sysname
AS
BEGIN
    EXECUTE AS Login = @LoginName;

    SELECT [name] AS GroupName,
           usage AS Usage
    FROM sys.login_token 
    WHERE [type] = N'WINDOWS GROUP';
END;
GO

When I executed this on one of my SQL Server 2025 systems this way:

EXEC dbo.GetGroupsForALogin N'GREG7680V2\Greg';

It returned the following:

Note that the Usage column could also return DENY ONLY or AUTHENTICATOR.

In an ideal world, every event in a data stream would arrive exactly when it was generated and in perfect chronological order. But in reality, streams are messy. Events can arrive late, out of sequence, or even replayed after retries. This is a normal characteristic of distributed systems — not a failure — and it’s something we need to design for in any real-time analytics pipeline.

There are several reasons why events might not arrive in order. Sometimes there’s network latency between devices and the ingestion point. Other times, devices might buffer data locally and send it in batches when a connection becomes available. In cloud or IoT scenarios, retries or transient service interruptions can also cause duplicates or delayed messages. So, when you’re analyzing a stream, the order of arrival isn’t always the same as the order of occurrence.

If we don’t handle this correctly, our analytics can become inaccurate. For example, imagine calculating the maximum temperature per minute from a stream of sensor readings. If a delayed event arrives after that minute’s calculation has already been finalized, the true maximum might never be reflected. Or in financial transactions, a late-arriving refund event might be missed from a daily total, making the metrics misleading.

To deal with this, KQL provides tools and patterns for handling out-of-order events.

The most common approach involves windowing with tolerance. You define a time window — such as one minute, five minutes, or one hour — and then specify a grace period or lateness tolerance that tells the system how long to wait before finalizing results. For example, you might say: Process events in one-minute windows, but allow up to 30 seconds for late arrivals. This ensures that slightly delayed data is still included in the correct calculation, without holding up the entire stream indefinitely.

Another technique is to use event timestamps rather than ingestion timestamps for ordering and grouping. That way, even if an event arrives late, it can still be associated with the time it actually occurred. Fabric’s real-time processing and KQL both support this concept — ensuring that the logical order of events is preserved, even if their physical arrival order isn’t.

Handling late or out-of-order data is critical for accuracy in real-time analytics. Without it, aggregates, anomaly detection, and alerts can all produce misleading results. By defining proper windows, tolerances, and timestamp logic, we can make our analytics both robust and realistic, reflecting how data behaves in the real world rather than assuming perfect conditions.

Events won’t always arrive in order, and that’s expected. By using KQL’s support for windowing and lateness handling, we can still produce accurate, reliable insights even when the data stream is imperfect — which it almost always is.

Learn more about Fabric RTI

If you really want to learn about RTI right now, we have an online on-demand course that you can enrol in, right now. You’ll find it at Mastering Microsoft Fabric Real-Time Intelligence

I recently received a review copy of Extreme DAX: Take your Power BI and Fabric analytics skills to the next level by Michiel Rozema, Madzy Stikkelorum, and Henk Vlootman from my friends at PackT.

Authors

Michiel Rozema is a long-term IT industry veteran. He was the data insight lead at Microsoft Netherlands and launched Power BI in the country. Michiel works as a data analyst and architect for his company Qanto. He is a fellow Data Platform MVP.

Madzy Stikkelorum works with Michiel at Qanto. She has written on DAX and speaks at international conferences.

Henk Vlootman runs the consultancy firm Qanto with Michiel. Originally working with Excel, he moved his focus to BI and data analytics when he saw Power Pivot. He now specializes in Power BI and Fabric, is a founder of the Dutch Power BI user group, and is another fellow Data Platform MVP.

It’s also worth calling out Terence Brown and Jeffrey Wang who were reviewers for this book. Jeffrey was a co-inventor of DAX. Both are well-positioned to be technical reviewers for the content.

Content

DAX is a curious language. I often tell students that even those of us who have worked with it ever since it first appeared, still dabble with it. I know only a handful of people who really think in it.

For that reason, what most people need isn’t detailed coverage of the rules of the language. Instead, they need to understand how experts think about solving problems with DAX. I’ve always said I like cookbook style books on languages like DAX, and before it, MDX.

I like the way that this book shows you how real problems are solved.

Keep in mind though, that DAX is often only one way to solve many problems. So often, the problem is better solved upstream before the data gets to where you’re using DAX. I’m often telling clients that if you find yourself struggling to write complex DAX, it probably means that your underlying data model isn’t very good. If you fixed the data model, the DAX required becomes simpler.

And given how DAX is still evolving, I particularly liked seeing how up to date the coverage of the DAX language in the book is. A good example is the content on time intelligence that covers defining custom calendars, and content on calculation groups. But I think the best example is the number of places where examples that I remember seeing in the first edition are now implemented very differently in this second edition.

One challenge with up to date content is that it often depends on the use of preview features, and the authors call this out in places. This book also helps you to follow along by providing downloadable example code files.

Performance isn’t ignored either. There is good content on using performance analyzer and how DAX queries interact with populating report visuals, along with a detailed discussion on how to use window functions for benchmarking.

The book is very readable. This is always a tribute to any group of authors, but particularly those for whom I presume English is not their native language.

Some different ideas

With any book that covers aspects of data modelling, there are always things I see where my reaction is that I wouldn’t do it that way and there are some in this book. They aren’t wrong, but I’d rather see it done differently.

An example is a model that had a Date table, but also had Customers, Products, and Cities tables. Note the last three are plural and the first one is singular. There was also a table called fSales. I’m not a fan of prefixes. (And the same applies to those who use Dim and Fact as table prefixes). Similarly, there are columns with names like InvoiceDate (no space) but also columns with names like Month Name (with a space).

I prefer everything to be human readable, and ready to be used in reports, etc. without needing to rename everything. But that’s just me. Not everyone agrees. And even the DAX language is frustrating in that regard, with reserved words like ISCROSSFILTERED and USERPRINCIPALNAME where they just string together many words, without an underscore or anything to separate them. Mind you, the SQL Server team is worse on that as they sometimes separate them, and other times don’t.

These are minor comments in relation to this book though, and DAX is the primary topic, not data modelling.

Summary

Overall, this is an excellent book. It isn’t for beginners. It assumes you have some background with using DAX. Even better if you’ve been trying to use DAX but become frustrated. I particularly liked the practical nature of the content, compared to books that target more theoretical topics. It focusses on how and why more than on what.

The coverage of how to use the language is really detailed, easy to follow, and up to date. You can’t ask for much more in a book.

9 out of 10

I’ve mentioned that in Microsoft Fabric, both KQL and SQL are available as query options — and they each have their strengths. So when should we use each?

KQL, or Kusto Query Language, is purpose-built for analyzing logs, telemetry, time-series, and streaming data. It’s optimized for massive volumes of events, and it’s designed to help you spot trends, anomalies, and behaviors over time. If you’re working with sensor readings, application traces, or event streams, KQL is the natural fit.

SQL, on the other hand, excels with structured, relational analytics — the kind of work you might do when querying data warehouses or relational tables. It’s familiar to most data professionals and works beautifully for joins, aggregations, and reporting-style queries.

The great thing is that KQL databases support both languages. You don’t have to choose one forever — you can use whichever fits your background or the problem at hand.

Let’s see a few examples.

Note: the line wrap shown in KQL was just to make it displayable side-by-side, not for real code.

If your team already knows SQL, start there. It’s accessible and productive right away. Then, as you explore more advanced time-series or streaming analytics, you can move into KQL to take advantage of features like time windows, joins across streams, and pattern matching.

Ultimately, the choice isn’t about one being better — it’s about what’s right for your workload and your team. In practice, many organizations use both: SQL for structured analysis and KQL for live, real-time insights.

Learn more about Fabric RTI

If you really want to learn about RTI right now, we have an online on-demand course that you can enrol in, right now. You’ll find it at Mastering Microsoft Fabric Real-Time Intelligence

This is more for my own reference than anything else but I needed to enter a backslash key while using a US keyboard but with UK keyboard settings.

After trying pretty much every key on the keyboard in all combinations, I realised there was no key combination that would do this directly. Curiously it’s a common problem and I found untold blog and forum entries that were not helpful. They basically said to change your input to US when using a US keyboard.

I figured there is always a way of entering a character by its code and of course this works. If you have a keypad, you can enter Alt92 and you’ll get a backslash. That’s all I needed in this case.

KQL, or Kusto Query Language, is the primary way we interact with data stored in a KQL database. It’s a read-only, declarative language — meaning you describe what result you want, rather than giving the system a sequence of procedural steps to perform. It’s designed specifically for interactive analytics on large, fast-moving datasets such as telemetry, logs, or IoT events, where you need quick insights rather than data updates.

One of the biggest differences you’ll notice compared with SQL is the pipeline syntax. Instead of writing long, nested queries with sub-selects or CTEs, KQL uses the pipe operator (|) to pass the output of one operation directly into the next. This creates a kind of query “flow” — very readable and very modular. For example, you might start with a table name, filter the rows, then summarize, and finally sort the results, each step connected by a pipe. It reads much more like a logical sequence of transformations than a single dense statement.

KQL is particularly strong when you’re working with time-series data. Time is usually the first dimension you filter on.

You can easily select events that occurred in the last hour or the last 24 hours using simple expressions like where Timestamp > ago(1h). Then you can bin the data into time intervals — say, five-minute or one-hour buckets — using the bin() function. This allows you to calculate aggregates over each window, such as counts, averages, or maximum values. It’s a natural fit for telemetry and monitoring workloads where you want to see how something changes over time.

The language also provides a wide set of operators for pattern detection and anomaly analysis. You can use functions like make-series, join, or extend to correlate data across streams, or apply statistical and machine-learning style operators to identify unusual values. This makes KQL well suited not only for dashboards, but also for alerting and diagnostic analysis — for example, spotting sudden spikes in error rates or detecting when a device stops sending signals.

Because KQL is read-only, it’s very safe to use even on production datasets — you can’t accidentally modify or delete data. The engine is optimized to scan large amounts of data quickly, so results typically come back in seconds even over billions of rows.

KQL gives you an efficient and expressive way to query, filter, and summarize streaming or historical telemetry data. Its pipeline syntax keeps complex analyses understandable, and its time-based functions make it an ideal language for exploring trends, detecting issues, and understanding what’s happening in your systems in real time.

Learn more about Fabric RTI

If you really want to learn about RTI right now, we have an online on-demand course that you can enrol in, right now. You’ll find it at Mastering Microsoft Fabric Real-Time Intelligence

I often need to create a T-SQL script that reliably drops and recreates a database each time it is executed. A common requirement is that I’m building a script to test come concept and I want to just re-run a script to get back to a known point after cleaning up all that I’ve done before. I could even just want to create a database in a particular way, and I want to make sure it doesn’t already exist.

Unfortunately, reliably dropping a database in a script that you run from SQL Server Management Studio (SSMS) is harder than it looks. And that seems silly.

In the documentation at Microsoft Learn, the article for DROP DATABASE says:

You cannot drop a database currently being used. This means open for reading or writing by any user. To remove users from the database, use ALTER DATABASE to set the database to SINGLE_USER.

Based on this, I had previously been using the following style of script to drop and recreate a database:

USE master; 
GO

IF EXISTS(SELECT 1 FROM sys.databases WHERE name = N'Blah') 
BEGIN 
    ALTER DATABASE Blah SET SINGLE_USER WITH ROLLBACK IMMEDIATE; 
    DROP DATABASE Blah; 
END; 
GO

CREATE DATABASE Blah ON
...

However, what I’ve found is that when I execute this from SSMS, it doesn’t always work. Fairly randomly (and hard to reproduce), it fails with an error message telling me that the DROP failed because the database was in use. But isn’t that the whole purpose of setting it to single user?

The problem

The problem seems to be that although the database gets set to single user, another connection could be made to it before the DROP DATABASE statement occurs. Setting the database to SINGLE_USER isn’t enough as the current database context for the script is the master database, not the database in question.

A number of users and fellow MVPs have told me that they’ve experienced the same issue. What we suspect is causing this is the Intellisense system within SSMS is connecting to the database to check the syntax of objects later in the same script. The problem only really seems to happen in SSMS but it could happen elsewhere.

A great suggestion from Rob Farley was to set the database OFFLINE instead of SINGLE_USER. While that would stop the other connection, the problem is that when you DROP a database that is OFFLINE, the files for the database don’t get deleted. You’d then need to have some additional script (nasty) to delete the files as well after the database was dropped.

You can get around this issue by resorting to dynamic SQL but there’s no way that should be required.

Writing a script to reliably drop and recreate a database is a pretty basic requirement, and I’ve not had that issue with other database engines.

What is really needed to get around this race condition is:

DROP DATABASE WITH ROLLBACK IMMEDIATE;

It just needs to be drop the database and if anyone is connected to it, disconnect them first.

I’ve added this to the SQL Server feedback sites multiple times over the years, but they seem to nuke all these entries as part of some clean up every few years. I wish they wouldn’t.

I recently mentioned that the AI Edition of the Data Science Summit is coming in June and that this time, I’m presenting two sessions.

The second session is Calling Ollama Based Text Embeddings Models from SQL Server 2025, and I’ll be discussing how the AI features that have been added to SQL Server in 2025 can be used with text embeddings coming from Ollama-based text embeddings models running locally on your servers.

Use the code DSSAI26SP20 to get 20% off Standard or Exec passes.

See you at DSS!

Find more details about the event here: https://ml.dssconf.pl/

It was great to catch up with Jerry Nixon today and to have him on a SQL Down Under podcast.

I recently had Jess Pomfret on a show, discussing Data API Builder. And the logical extension of that is to talk about SQL MCP Server.

Jerry is a Principal Product Manager for Data and AI at Microsoft.

He says he is focused on the developer persona to minimize onboarding friction and maximize value.

Jerry is leading efforts to integrate proven open-source tools and managed resources as drop-in components that developers can use to simplify emerging AI solutions.

He is also a professor of Computer Science and a regular speaker on .NET and SQL, with a focus on intelligent apps powered by practical architectures that really work.

And of course, Jerry says he’s a big fan of Data API builder.

I hope you enjoy our podcast. You’ll find this show (and previous shows) here: SQL Down Under Podcast

Also in June this year, the AI Edition of the Data Science Summit is coming. And this time I’m presenting two sessions.

The first session is Using AI Features in SQL Server, and I’ll be discussing the AI features that have been added to SQL Server in 2025.

Use the code DSSAI26SP20 to get 20% off Standard or Exec passes.

See you at DSS!

Find more details about the event here: https://ml.dssconf.pl/