Book Review: The Orange Book of Machine Learning

I recently received a review copy of The Orange Book of Machine Learning: Green Edition by Carl McBride Ellis from my friends at PackT.

Author

Carl McBride Ellis wrote this book as a Green Edition of the material he had been teaching across a whole range of Spanish cities.

Content

This is the sort of technical book that feels less like a formal textbook and more like spending a few very productive days with an experienced instructor. That makes sense: the book grew out of a five-day course, and its examples are built around Python, pandas, scikit-learn and Jupyter notebooks. The result is practical, opinionated and surprisingly personable. The language is quite precise but a little less conversational than I would have liked. It’s very to the point.

I did have a chuckle when I saw it was called the Orange book but was then Green. It took a moment for the reason to hit me.

I spent a lot of time on machine learning over the years, and this book covered a lot of very familiar territory, even though I’m rusty on much of it. It focuses firmly on supervised learning with tabular data. It starts with statistical foundations, exploratory data analysis and cleaning, then moves through cross-validation, regression, classification, ensembles, hyperparameter optimisation, feature engineering and tabular foundation models. That order works well because Carl does not rush straight to the glamorous algorithms. He spends time on missing values, leakage, scaling, metrics and uncertainty i.e., the things that usually determine whether a model is genuinely useful or merely looks good in a notebook.

What I enjoyed most is the author’s willingness to take a position. He didn’t just present a neutral catalogue of algorithms, which would be really easy to do. Carl warns that repeated hyperparameter searching can overfit the validation set, questions the value of some common practices, and repeatedly argues for simple, robust models rather than unnecessary complex ones. He’s also to the point about imbalanced classification, describing undersampling as wasteful and oversampling as fundamentally questionable. You might not agree with every verdict that he presents, but the strong opinions make the book memorable and encourage you to think rather than simply copy recipes.

The explanations are usually compact and intuitive. I liked the description of overfitting, and the way it’s presented as learning the noise after the signal has already been captured, which is a much more useful mental picture than treating it as a vague warning. The book also frequently connects theory back to code, and the many diagrams help make abstract ideas such as gradient descent, calibration, ensembles and transformer architectures easier to grasp. There is enough mathematics to show what is happening underneath, but most sections get quickly back to implementation.

I also liked the character shown in the book. Chapters start with quotations, the olive-themed design gives the book a recognisable visual identity, and small asides such as turtles all the way down when discussing surrogate models, were very familiar to me. And they stop the material from feeling sterile. The linked GitHub resources are another real strength, because this is a book to read beside a laptop rather than passively from the sofa. Even when Craig is being mathematically precise, he generally sounds like a teacher anticipating the question you were about to ask, or the mistake you might make.

That said, compact occasionally becomes compressed. I’m worried that some topics arrive with equations and terminology at a pace that might scare off a beginner. The breadth is impressive, but it also means that subjects such as generalized additive models, conformal prediction and foundation models sometimes feel more like guided introductions than full treatments. The recommended-reading sections do help, although readers wanting deep proofs, production deployment or end-to-end engineering guidance will need other books.

I really liked the final chapter. It’s a great addition to the book. Rather than pretending neural networks automatically win everywhere, Craig explains why traditional multilayer perceptrons often struggle on tabular data and introduces newer models such as TabPFN and TabICL. This keeps the book current without losing its central message about understanding your data, evaluating honestly and choosing models for evidence-based reasons.

Summary

This book is an energetic, practical and refreshingly candid guide. It is best suited to readers who know a little Python and want to understand not only how to fit models, but how to avoid fooling themselves. It is not effortless reading though, yet I think it’s sufficiently approachable to be useful and full of the sort of advice that should stick.

8 out of 10

2026-07-31