Since I posted some StreamInsight info the other day, I've had a bunch of people asking me what StreamInsight is used for.
StreamInsight is Microsoft's implementation of Complex Event Processing. This is not a new market but it is new territory for Microsoft.
Complex Event Processing (CEP) is all about querying data while it's still in flight. Traditionally, we obtain data from a source, put it into a database and then query the database. When using CEP, we query the data *before* it hits a database and derive information that helps us make rapid business decisions, potentially also including automated business decisions.
I liked the way that one of our new colleagues (Sharon Bjeletich) put it to me: "It's about throwing the data at the query, rather than throwing the query at the data".
There are lots of places that this makes sense but they all involve relatively high data rates. Good examples of these are automated trading in capital markets, fraud detection in networks or in casino operations, battefield control systems for military use, outbreak management for public health, etc.
While StreamInsight may combine the data with reference data stored in SQL Server, the primary development skills needed for working with it are .NET development skills.
Interesting, but I am not really sure I see the need for such system. It sounds like it is used to "check something and do some decision" on, for example, an INSERT statement that is being executed against the database. I have the feeling that this approach is completely wrong and that such feature is completely unnecessary. If you want to "check something", then you should do that before executing the query. If you cannot do that, then you should be executing the query against a temporary table and then "check something" on that temporary table.
It's another one of those systems that I'll do my best never to use.
When you say you should "check it before executing the query", I suspect you're assuming that the query is a database query. It's not. It's a standing LINQ query in .NET. This is done well before it goes anywhere near the database. The data is being queried in-flight in a .NET layer.
I think Giammarco's question is indicative of what we're going to see a LOT of: the fact that this product is being released as part of SQL Server is going to confuse a lot of people. This probably should have been released as a separate product or through some .NET channel. Although perhaps there is no appropriate ship vehicle on that end? Maybe part of Windows itself would have been better?
+1 with Adam.
Why is this bundled with SQL Server at all?
I think Visual Studio would have been the correct channel for shipping it but I imagine it's gone into SQL Server as it's a channel that can be monetized. (Just my guess)
Thanks Adam… I was really going nuts thinking I was missing some crucial aspect of what StreamInsight was all about… I just could not see the SQL Server connection!