I love working with Azure Data Factory (ADF). There are times though, when I’m stuck wondering what’s going on, and it takes me a moment to discover it.
One that had me puzzled recently was a Copy activity that just wouldn’t seem to start. The pipeline had started ok. Previous activities in the pipeline had run just fine. But when it got to the Copy activity, it said “Queued” and just kept saying that seemingly endlessly.
2021-01-06
I often need to create a list of columns for a table. Sometimes it’s because I’m generating scripts but there are other times that I need it for dynamic code.
Let’s use WideWorldImporters as a sample. Imagine I need to get a list of columns for the Sales.Orders table.
I’ve seen people doing this the wrong way. They try to use a SELECT statement that joins a variable to itself like this:
2021-01-05
The next set of introductory T-SQL topics that I want to talk about is how to change data and/or data types. The first issue is dealing with NULL values.
Once you get your head around the idea that NULL means the lack of a value (NULL isn’t a value), you might need to replace NULL values with something else. While you could just do that with a CASE statement, T-SQL provides two built-in functions for doing this.
2021-01-04
Hi Folks,
Just wanted to make a post to wish you all a happy new year for 2021.
I doubt there’s anyone much on the planet who didn’t have a very, very peculiar, or very, very difficult 2020, and while there’s a glimmer of hope with the virus now as vaccines arrive, I suspect that 2021 is still going to be dominated by the coronavirus, and a very tough year for so many.
2021-01-01
I was really excited today to see details of the upcoming #dataweek. Anything with a name like that already has my interest.
In this case though, there’s a #CreateData event that’s happening as part of #dataweek. And the focus of that looks to be on Azure Synapse Analytics.
I know that many of my readers won’t have spent much time with Azure Synapse Analytics. The product has grown out of Azure SQL Data Warehouse (note not Azure SQL Database) but it’s now much more than just an upgraded Azure SQL DW.
2020-11-25
I was really pleased to get to record a SQL Down Under podcast with an old friend Pedro Lopes recently. Pedro is a Principal Program Manager with Microsoft in the Azure Data SQL Server area.
A year or so back, I had Joe Sack on the show telling us where Intelligent Query processing was heading, and Pedro now fills us in on where it’s got to.
We recorded the show earlier in this month but I couldn’t release it until after the PASS summit where some of the features were announced.
2020-11-23
I work with a lot of technology on a daily basis, much of it amazing. But I still get excited when relatively small enhancements are made, and they make my life or development much better.
Timezone support in schedule triggers in Azure Data Factory is one of those changes.
In Data Factory, the most common type of trigger that we use is a schedule trigger. It’s used to run a pipeline at a specific time or series of times. But one of the most painful aspects of these triggers is that they didn’t have time zone support.
2020-11-04
When I’m working in client sites, I get really concerned when I see personal data not being handled or protected appropriately. And one of the biggest sins that I see is where developers have pretended to be encrypting data, but they really aren’t encrypting it.
I’m sure that looks good for a manager but please, just don’t do this !
When I look at the table definition shown in the main image above, my eye is drawn to the column called EncryptedTFN. In Australia, we refer to our tax numbers as TFNs. They are 11 digits long for most people and should never be stored in plain text in a database. The column should be encrypted.
2020-10-27
I’m pleased to let you know that version 20 of our free SDU Tools for developers and DBAs is now released. It’s all SQL Server and T-SQL goodness.
If you haven’t been using SDU Tools yet, I’d suggest downloading them and taking a look. At the very least, it can help when you’re trying to work out how to code something in T-SQL. You’ll find them here:
https://sdutools.sqldownunder.com
Along with the normal updates to SQL Server versions and builds, we’ve added the following new functions:
2020-10-22
Once again today, I was asked by a developer, how you can “turn off those annoying completion time messages in SSMS”.
A few releases of SQL Server Management Studio (SSMS) back, the product team decided that we’d all like completion times shown in the Messages output tab. I don’t share their enthusiasm for them.
For a start, a completion time was already shown here in the Properties window for the query, along with much more info:
2020-10-21
