There’s a UK game show that I like to watch. It’s called Lingo and it’s the standard word game turned into a full TV show. It’s based on a US game show of the same name.
While watching it, I realized that I’ve come to form an immediate opinion on the education level of the contestants, not based on their appearance, not based on their accents, not based on their assortment of tattoos, but on how they pronounce the letter H.
2026-09-07
Streaming systems produce large and continuous data flows, so it’s essential to have a clear data retention strategy. Without one, storage costs and data management complexity can increase very quickly.
The first decision is what type of data to keep. In many cases, raw event data is valuable for troubleshooting or forensic analysis but may not need to be retained indefinitely. Aggregated or summarized data (for example, hourly or daily averages) is often sufficient for long-term trend analysis.
2026-09-06
I recently read the 2nd edition of a book on serverless ETL. Serverless ETL and Analytics with AWS Glue (2nd Edition) . I hadn’t had the chance to read the first edition of this book, but my friends at PackT sent me the 2nd edition to check out.
Noritaka Sekiyama is a big data engineer working at a data and AI company.
Albert Quiroga is a senior solutions architect at Amazon.
2026-09-05
In practice, most real-time systems use a combination of streaming and batch processing rather than relying entirely on one or the other. This approach is often described as a hybrid architecture, where each path serves a different purpose.
The hot path (also called the streaming path) handles data as it arrives. It’s designed for immediacy, enabling rapid decisions such as triggering alerts or applying automated actions. The trade-off is that it usually works with limited context and may rely on approximations or rolling summaries.
2026-09-04
I’ve never liked how the LEN function works in T-SQL for SQL Server.
To test out what others thought, I recently posted a short quiz on my social networks. Here’s the question:
What would you expect the output from this query to be?
DECLARE @Value1 varchar(10) = 'Hello ';
DECLARE @Value2 varchar(10) = 'There ';
SELECT LEN(@Value1), LEN(@Value2), LEN(@Value1 + @Value2);
Nothing tricky. The first value has 3 trailing spaces. The second value has one.
Then I gave the following answer options:
2026-09-03
Monitoring brand reputation has become a continuous challenge in the age of social media. Platforms like X, Facebook, and Reddit can amplify sentiment rapidly, and a single negative trend can reach large audiences within minutes.
Traditional sentiment analysis systems are often designed for batch processing — for example, analyzing posts every few hours or once per day. While this can capture overall trends, it fails to provide early warnings. By the time a report shows a spike in negative sentiment, the situation may already be public and difficult to control.
2026-09-02
I recently listened (via Audibla) to Timothy Snyder’s book On Tyranny: Twenty Lessons from the Twentieth Century . I wasn’t sure what I’d make of it but I’d heard it was interesting, and I noticed that on a show led by Rachel Maddow.
Timothy Snyder is a well-known author, historian and intellectual. He’s an expert on eastern Europe and on the Second World War.
I decided to go through this book for two reasons. I was fascinated by his thoughts on tyranny, and I liked the fact that it was a short book. I’m also a fan of books that are narrated by their authors, and Snyder is the narrator for this book.
2026-09-01
IoT environments typically involve large numbers of devices — sensors, controllers, and machinery — each continuously generating telemetry data. Common examples include temperature, vibration, pressure, voltage, and power consumption readings. These data streams provide valuable insight into device health and operating conditions.
When a component starts to degrade or operate outside normal ranges, early signs often appear in the telemetry before a full failure occurs.
The challenge is that traditional reporting systems tend to work on batch cycles — for example, generating summaries every hour or day. That delay means early warning signs can be missed, especially in high-frequency data streams where conditions can change in seconds.
2026-08-31
I recently mentioned that I’ve spent some time lately reading some older and more famous books. I’m particularly keen to re-read books that I remember strongly from when I was young. I love John Steinkbeck’s books and Of Mice and Men particularly stuck in my mind.
John Steinbeck was an American writer and novelist who won the 1962 Nobel Prize in Literature for his realistic and imaginative writings. I’ve read many of his books and have loved them.
2026-08-30
Credit card fraud continues to be a major challenge for financial institutions. It can happen within seconds—often before traditional systems even notice. Once a fraudulent transaction is processed, recovering the funds is difficult or impossible.
Most legacy fraud detection systems rely on batch processing, where transactions are analyzed in hourly or daily cycles. That delay makes them ineffective for real-time protection.
The business need here is clear: detect and respond to fraud as it happens. This means evaluating transaction streams in real time, identifying anomalies within seconds, and automatically taking action—such as flagging or blocking suspicious activity.
2026-08-29