At times, the task of understanding the contents of an image has used two sequential steps of segmentation and recognition, without interplay between the two. A better, though more complex, approach is to determine object boundaries as part of the process of recognizing those objects, that is, “joint image segmentation and shape matching.” A priori information about the shapes of expected objects is generally used, but these priors can be either overly simplistic or demanding to apply.

Schmitzer and Schnörr describe an approach that achieves a higher level of generality yet can be applied in computationally reasonable ways. The essence is to define a function that can be simultaneously optimized across different poses and variations in segmentation. The set of possible “transports” defines a form of a 2-Wasserstein space, within which poses and segmentations form a manifold. This allows for simultaneous optimization, though there is some low-level approximation (in terms of rotation and scale, for example). Practical use of the technique requires the application of several computational methods, from contour learning to the formulation and solution of a min-cut graph problem to find the optimal transport. Each is discussed in detail and external works are cited.

One interesting section discusses the authors’ strategy for dealing with the following problem: the natural technique considers all variation from the prior as having equal weight, but some object deformations are more likely than others. To deal with this, they describe a family of possible object shapes. This family is subject to statistical analysis to find the principal components of the variation, in the contour representation, which can be related directly to the optimization problem.

The technique is powerful, yet mathematically complex. This paper is not simple to read and digest only because of the mathematical fluency required to understand and to follow the development of the technique. It is very well written; two obviously talented authors have done a great job of making the imposing mathematics involved as clear as possible. Still, not all readers will be able to follow each step, particularly at the first reading. Importantly, the references are comprehensive and especially serve to provide background reading on the mathematics involved. This is a good approach for the solution of a difficult, common, and fundamental problem: finding objects in unsegmented images. A complex solution to a difficult problem is not surprising; the investment of time to understand the mathematics involved is easily justified. This paper provides a thorough and clear description of such a solution, and is recommended to anyone interested in image segmentation and understanding.