Parallel Programming In TSQL: Is It In Our Future?

I saw an interesting videopod from Steve Jones the other day where he noted that upcoming processors would have many more cores than now and hoping that DBAs won’t have to learn to write multiprocessor-style code. What was also interesting was the way he described that he learned some of this style of coding but later when he came back to it, he realised how much he thought he knew but didn’t.

For languages like T-SQL, we don’t have inherent support for multi-threading. In fact, the only trace I can see of this in T-SQL today is the ability to have multiple readers on a service broker queue. In general, we haven’t needed this because SQL Server systems constantly have many requests thrown at them concurrently anyway and there is a natural style of parallelism happening.

But if languages need to evolve, I think it’s important that wherever possible, that it’s the languages that work out how to parallelise the code, not the developer. I often mention to people that in one interview I did with the late Dr Jim Gray, he noted that people learning to write multithreaded code tend to go through three phases:

1. when it all just looks like magic and you don’t understand any of it.

2. when you think you understand it.

3. when you get wise.

That’s very insightful and has stuck with me ever since. I’ve written a bunch of multithreaded code over the years and my experience exactly parallels (pun intended) the phases he described. Getting it right is much, much harder than it looks.

2008-07-21