Evaluating derivatives of functions is a problem frequently encountered in various areas of scientific computing. This is especially important if the task at hand comes from a nonlinear problem, regardless of whether the precise background is in optimization, control theory, or differential equations. Thus, there is a need for well-understood and efficient methods for the evaluation of derivatives.

A traditional approach is the technique of numerical differentiation. However, it is well known that this method can suffer from serious numerical instabilities, due to the cancellation of terms in finite-precision arithmetic. A conceptually different idea that has the potential to avoid such problems is symbolic differentiation by computer algebra packages, but this approach tends to be too slow for today’s large real-world applications. Thus, it seems that a completely different path may be worth studying. The topic of this book is such a path: the method of algorithmic--or, as some authors prefer to say, automatic--differentiation.

Written by some of the leading researchers in this area, the book provides very detailed information about many aspects related to algorithmic differentiation. The style of writing is clear and comprehensible for novices.

The text is divided into three parts. The first part (chapters 1 to 6) is a comprehensive introduction to algorithmic differentiation. It can be used, without any additional material, to learn the basic ideas and the different concrete approaches (in particular, the forward mode and the reverse mode of algorithmic differentiation), including their advantages and disadvantages with respect to computational complexity and memory requirements; it also explains the issues that need to be addressed when implementing such algorithms. This part consists mainly of well-established, classical, standard material.

The second part (chapters 7 to 11) deals with more advanced questions and provides more recent results. The emphasis here is on the computational complexity of the problem and on efficient ways to minimize the run time of the algorithms. Readers who only want to learn what algorithmic differentiation is and how it can be used can skip this part, but it makes for interesting reading for those who want to dig deeper into the matter and use algorithmic differentiation in practice.

Finally, the third part (chapters 12 to 15) addresses topics that are currently under investigation by the algorithmic differentiation community. This part is of a very advanced nature and is mainly of interest to people who want to undertake research in this area, including those whose problems cannot be handled appropriately by standard algorithms because, for example, they are very large or require derivatives of a high order.