A dataset, generically speaking, is a collection of numbers. It constitutes an information element when structure and context are assigned. The objective of machine learning (ML) is to interpret an information element as a human would. This requires further abstraction through a prior knowledge filter. This requires the context-sensitive filtration of information according to a preset algorithm, thus requiring a set of complex and convoluted decisions for ML procedures, including algorithm selection for learning multiparameter models. Hence, it is difficult to arrive at a universally acceptable definition for automated ML.

The editors state in the preface:

The field of automated machine learning (AutoML) aims to make these [procedure design] decisions in a data-driven, objective, and automated way: the user simply provides data, and the AutoML system automatically determines the approach that performs best for [a] particular application.

The book has three parts: “AutoML Methods,“ “AutoML Systems,” and “AutoML Challenges.” Part 1 has three chapters. Chapter 1, by Feurer and Hutter, discusses hyperparameter optimization (HPO), which is required to set up learning models such as an AutoML framework, deep neural networks, and optimization to minimize human efforts. The authors discuss prominent HPO methods such as the black-box function (without model) and Bayesian optimizations, which require large computational resources. Next, they discuss multi-fidelity methods that require lesser resources. The chapter closes with 160 references and comments on HPO challenges and future research directions.

In chapter 2, Vanschoren provides an overview of the state of the art in metalearning, or learning to learn. It has 191 references. A survey of learning methods in the context of a problem helps with the selection a faster learning method. Chapter 3, by Elsken et al., gives an overview of the current research in automated neural architecture search methods and categorizes them by search space, search strategy, and performance estimation. This chapter has 75 references and concludes with comments on future directions.

The second part discusses six easy-to-use AutoML systems, including a critical evaluation in terms of their performance. Chapter 4, by Kotthoff et al., discusses how Auto-WEKA uses an algorithm based on Bayesian optimization for model selection and optimizing hyperparameters. It includes 33 references and is upbeat about Auto-WEKA performance. In chapter 5, Komer et al. describe the Hyperopt-Sklearn software package that automates the selection of search space and standardizing components of the learning machine. Using a benchmarking dataset, the authors demonstrate that the search space is effective and practical.

Feurer et al., in chapter 6, present Auto-sklearn, based on a Python machine learning package that uses 15 classifiers, 14 feature preprocessing methods, and four data preprocessing methods that yield a structured hypothesis space with 110 hyperparameters. Auto-sklearn is shown to be better than other AutoML methods.

In chapter 7, “Towards Automatically-Tuned Deep Neural Networks,” Mendoza et al. present two versions of Auto-Net software that provide automatically tuned deep neural networks without any human intervention. The authors are encouraged by the results of the trials.

In the next chapter, Olson and Moore present TPOT v0.3, a genetic programming-based software package for tree-based pipeline optimization of AutoML. TPOT has been benchmarked against a series of 150 supervised classification tasks; the authors are satisfied with the results.

The last chapter in this part, by Steinruecken et al., discusses a project with the aim to automate data science, producing predictions and human-readable reports from raw datasets. Further, they describe what is thought to be the common system architecture, required design decisions, and technical challenges of an automated statistician system.

In the only chapter in the third part, Guyon et al. present “Analysis of the AutoML Challenge Series 2015–2018.” It summarizes the authors’ impressions on the state of the art of AutoML, gained by organizing a couple of competitions in which the challenge was limited to “supervised learning,” “feature vector representation,” and “less than 200 Mbytes of homogenous datasets for training, validation, and tests,” while putting a cap on computing resources. These challenges identified issues and difficulties in respect to “categorical variables,” “missing data,” “a large number of classes,” “regression,” “sparse data,” “large datasets,” “presence of probes,” “type of metric,” “time budget,” and “class imbalance.” The chapter has a bibliography with 70 entries.

This interesting collection should be useful for AutoML researchers seeking an overview and comprehensive bibliography. At first it was a bit difficult to comprehend the scope of the book despite a well-written preface--perhaps a formal introduction is required.

More reviews about this item: Amazon