There is an increased interest in machine learning (ML) in both academy and industry. One consequence is a greater demand for textbooks and other educational materials that explain the different facets of ML to a heterogeneous readership. While many texts cover ML methods, Masters’ book focuses on assessing and improving predictive models, two really hot issues that make a difference when ML is used in the real world. Like his other books, the style of presentation is light and not too formal, and Masters walks readers through the essence of things.

ML offers a plethora of methods for generating models from data. Also, the advent of new astonishingly effective methods like deep learning and related easy-to-use tools pushed ML outside academic and small industrial niches, making it possible to realize intelligent features almost everywhere. As a consequence of this transformation, the use of ML tools is now within reach of many professionals; therefore, a guide to the correct use of such tools is necessary. Putting aside high-level concerns such as ethics, transparency, and so on, one basic concern is how to guarantee the effectiveness of a predictive model. This is not a trivial issue, especially in real-world applications, where effectiveness may mean money, safety, or other demanding aspects. But assessing the effectiveness of ML models is not easy at all: assessing methods rely on sophisticated statistical theories and methods. If we consider that these models are being implemented outside academia to deal with real-world situations, this problem is further exacerbated. Different books are available in the literature that discuss the issue of assessing predictive models, but most of them are oriented toward an academic readership with the intent of rigorously reporting assessing methodologies and studying their theoretical properties. As a consequence, books like Masters’ are highly welcome in the panorama of technical literature.

Somewhat related to assessment is the topic of quality improvement. This can be accomplished by either optimizing the parameters of the ML models or intelligently combining models so that the quality of the combination is higher than that of a single model. Masters’ book deals with the latter, which is related to assessment by the fact that predictive models are seen as black boxes: apart from some general assumptions, any classification or prediction models can be considered for assessment and improvement. As for assessment, improvement must be done with a clear awareness of what has to be done: the risk of an oversimplified approach leads to poor results and wasted time.

This book lends a valid hand to understanding such difficult yet necessary arguments. It is focused on prediction and classification tasks only. Other types of tasks, for example, clustering, semi-supervised learning, and so on, are deliberately left out to avoid overwhelming the reader.

The book could be divided into two parts. The first part is devoted to assessing classification and prediction models. Chapter 1 is about numerical prediction and introduces via a critical discussion some classical topics like cross-validation. More interestingly, a lot of space is given to important questions like the computation of confidence intervals, which are considered as tools to provide strong estimations of the real quality of a model. The book also takes a quick glance at model consistency; however, more space should have been devoted to this topic, which is really important for real-world applications. Chapter 2 is about classification, where a lot of space is given to receiver operating characteristic (ROC) curves and related statistics. Also, confidence evaluation for classification through hypothesis testing is well explained. Like in every chapter, explanations are accompanied by simple yet effective examples. (In some cases, a lack of formalization or illustration is felt.)

Masters’ statistical background is easily recognized in the book. This is beneficial in many cases because other books sometimes overlook key questions (like confidence estimations); however, sometimes other important things are then left out. For example, I would have liked to learn more on consistency in numerical prediction. Furthermore, some widely used metrics for classification, such as precision/recall and F1 measures, are not given the space they deserve in a book like this.

Chapters 3 and 4 are dedicated to resampling for model assessment. Bootstrapping is introduced to estimate bias and variance, and then this method is specialized for estimating the quality of models through the well-known Efron’s E0 and E632 estimates. The reader is led step-by-step through this theory, avoiding complex formalization but focusing on the key concepts. (The included essential bibliography mentions more technical documentation, anyway.) Chapter 5 moves from the same argument of resampling to introduce bagging (which is soon abandoned for its limited efficacy) and boosting, more specifically AdaBoost.

What can be considered the second part of the book is committed to improving ML models through combinations of multiple models. Techniques are presented and compared on simple synthetic data to highlight their differences; this is very valuable information. Chapters 6 and 7 go ahead with improvement techniques. While chapter 5 shows a way to combine models through resampling (mainly, AdaBoost), in these chapters models are aggregated with some aggregation operator. In some cases, the theory behind the methods is too complex (as in the case of fuzzy integrals). The reader is then asked to make an act of faith and just believe that the theory is good--according to Masters, for practitioners, having a powerful tool is always a good thing, even if its workings are not completely clear.

Chapter 8 is about gating methods, that is, combination methods that use some features in data to decide which model to trust for the final decision. Finally, chapter 9 is about information theory and its role in variable selection. This is quite an outlier with respect to the other chapters; however, it does provide a clear view of the importance of Shannon’s mutual information as a necessary (yet not sufficient) quantity to achieve good models. More advanced material is outlined, for example, Schreiber’s information transfer.

Overall, this is a useful book for anyone who wants to build effective predictive models, since it describes powerful tools for effectively assessing and improving them in real scenarios. However, the included source code, sometimes filling several pages, is perplexing; it is unclear who this source code is for. This book is for people who want to solve problems with ML, using either off-the-shelf programs (which this book covers) or implementing them in their own language. For the latter, the companion website is more than enough. Wasting pages and pages on how cycles are done and what data structures have been used, after even more pages of C++ code (not a champion in self-documentation, as is known), interrupts really interesting sections.

More reviews about this item: Amazon