The Bit Bucket

Microsoft has had a strong lead in BI and analytics for a long time now. The introduction of the tabular data models in 2012 was a watershed moment.

Today, they have announced a private preview of Microsoft Fabric. It’s a bigger deal than the tabular model was, and will set the direction of BI tooling for a long time to come.

The opposition was already struggling to keep up, and in fact, hadn’t managed to do so. Microsoft Fabric will up the ante much, much higher than where we already were.

2023-05-24

I recently noticed a blog post from Jay Gordon that was calling out interesting content from Blaize Stewart on antipatterns for development with Azure Cosmos DB.

I thought this would be a perfect topic for one of our Cosmos Down Under podcasts and Blaize agreed to take part.

So much content talks about what to do with a product, and even though everyone knows there are things you shouldn’t do, very little content talks about these antipatterns. It was great to speak to Blaize about it, and to get his updated thoughts.

2023-05-17

I was recently doing work for a client where I needed to replace some Python code that was being used to upload a file attachment to a REST based service using a POST command. I was trying to do it using a Web Activity in Azure Data Factory. What was surprising was how little information I could find on how to do that.

While I can’t post all the code, I did want to post the core parts here in case it helps others, and for my own future reference. The basic steps involve:

2023-04-29

We have so many great tech friends in Poland. I was pleased to see they are running the MS Tech Summit, and pleased to be speaking at it.

Looks like a great set of sessions : https://mstechsummit.pl/en/ They tell me there are more than 100 sessions spread over 8 tracks.

Not in Poland? Not a problem. It’s running in-person and online, and with video on demand.

The early bird discount is still available but about to end.

2023-04-28

This one had me stumped lately.

I had an variable in ADF where the value contained carriage returns. I needed to remove them.

I guessed that would be easy:

@replace(variables(‘fileContent’), ‘\r’, ‘’)

But no matter what variation of this I tried, it wouldn’t remove them.

Thanks to the genius of fellow MVP Simon Sabin, the answer was:

@replace(variables(‘fileContent’),base64ToString(‘DQ==’),’’)

Note: if you need a line feed (LF) it’s base64ToString(‘Cg==’) and a carriage return / line feed pair (CRLF), is base64ToString(‘DQo=’).

2023-04-24

I was working with a client who has having trouble debugging an ADF pipeline, related to an ODBC linked service not working as expected.

The user had configured the connection string property of an ODBC connection this way:

•  Set a parameter to the linked service as ServiceDSN
•  Configured the connection string as @concat(‘DSN=’,linkedService().ServiceDSN)

The Test Connection for that worked fine, but when you click Apply, it fails with an Internal Server Error. Specifically, the error was:

2023-04-24

I was able to record another new Cosmos Down Under podcast today. My guest was Microsoft Senior Program Manager Rodrigo Souza.

In the show, we discussed the Change Data Capture feed for the Analytical store in Azure Cosmos DB. This is a powerful new capability and worth learning about.

I hope you enjoy it. You’ll find it, along with all Cosmos Down Under podcasts at:

Cosmos Down Under Podcast

2023-04-12

Over the last year, I’ve come to know Alex Kolokolov more, through involvement with his data visualization challenges. I was really pleased to see he’d written his first book Make Your Data Speak (Creating Actionable Data through Excel For Non-Technical Professionals).

Things I Liked

I really liked the conversational style of the book. It’s all structured around  an approach of “Let’s see how this happens by example”. The tone was really refreshing and should be good to hold people’s interest.

2023-03-12

In the part 1 of this series, I discussed the positioning of Change Data Capture. In part 2, I want to cover how it works.

Log Reading

There are many ways that you can output details of changes that occur in data within SQL Server. Many of those methods require actions to occur at the time the data change is made. This can be problematic.

The first problem with this, is the performance impact on the application that’s making the change. If I update a row in a table and there is part of the process that writes details of that change to some type of audit or tracking log, I’ve now increased the work that needs to happen in the context of the application that’s making the change. Generally what this means, is that I’ve slowed the application down by at least doubling the work that needs to be performed. That might not be well-received.

2023-03-06

I’ve worked in this industry a long time now.

From 1981 to 1986, I worked as an engineer for Hewlett-Packard. Yep, I’m “have actually met both Bill Hewlett and Dave Packard” old. They’d retired when I met them and John Young was in charge, but they came to Palo Alto to a shareholders’ meeting and I was there at the time.

Most of my work was on HP3000 commercial mini-computers. The level of engineering on those systems was unlike anything I’d ever seen either before or since. They were really quite superb.

2023-03-05