This is a wonderful handbook for multicore and graphics processing unit (GPU) programmers. Modern computing architectures have included multiple cores for nearly two decades. These parallel computing platforms require a new approach to software development. OpenMP, message passing interface (MPI), and CUDA are the main focus of this book. The author includes a number of illustrations and examples. Readers can gain much hands-on experience from reading this book.

It is organized into five parts. Part 1 includes chapters 1 and 2, which introduce the design of multicore software. Chapter 1 briefly goes over various types of computational architectures, each of which is introduced with its speedup and efficiency. Chapter 2 turns into an introduction to software designs of multicore programming. Since a computational task can be decomposed on multiple computing cores, readers will learn software designs and structure patterns for the decomposition.

Part 2 is on shared-memory programming, based on explicit and implicit parallelization. Chapter 3 is dedicated to explicit parallelization. One topic is threading in the explicit approach, and the other is synchronization mechanisms. The author further uses many examples to explain design patterns of producers-consumers and readers-writers. Chapter 4 covers implicit parallelization based on the OpenMP standard, which was originally designed for parallelizing existing sequential computations. The author not only teaches readers how to reduce development time based on OpenMP, but also addresses other difficulties in loop-carried dependencies.

Part 3 considers distributed memory programming in chapter 5. The main topic is MPI. MPI lays the foundation of parallel computing in multiple disjoint multicore machines. MPI can foster the development of portable and scalable large-scale parallel applications. The author focuses on several communication paradigms: point-to-point, collective, and one-sided. Further, the Boost.MPI library is introduced, so readers can use this library to simplify the use of MPI.

Part 4 is on GPU programming. Multicore GPUs were initially developed for the video gaming industry, but they have become standard components in modern computers. As such, multicore GPUs are programmed by many software developers for many other purposes, such as genomic sequencing, marketing research, and financial analyses. Chapter 6 introduces modern GPU programming based on Nvidia’s CUDA, with emphases on memory transfers, data placement, and thread execution. Chapter 7 covers high-level algorithmic approaches based on the Thrust template library, which can integrate both central processing units (CPUs) and GPUs.

The last part covers load balancing. The aforementioned topics are discussed with regard to homogeneous computing platforms only. However, there is a growing trend for heterogeneous computing platforms to play a role, especially in large-scale clusters. As such, possessing knowledge of balancing loads across different platforms will benefit software engineers. Chapter 8 discusses the modeling of load partitioning and data communications, and further introduces the Linda coordination language for static and dynamic load balancing.

In general, the book is well written. The author walks through the multicore programming technologies step by step. Various examples, diagrams, and sample codes are given to allow readers to get hands-on experience. This book will help elevate software engineers to a more sophisticated level, or it can serve as a textbook for undergraduate and graduate students.

More reviews about this item: Amazon