The problem of how information contained in multiple overlapping images of the same scene can be combined to produce images of superior quality is investigated in this book. This area, generically titled frame fusion, offers the possibility of reducing noise, extending the field of view, removal of moving objects, removing blur, increasing spatial resolution, and improving dynamic range. The research has many applications, in fields as diverse as forensic image restoration, computer generated special effects, video image compression, and digital video editing.

An essential step in frame fusion is image registration: computing the point-to-point mapping between images in their overlapping region. This sub-problem is considered in detail. A robust and efficient solution is proposed, and its accuracy is evaluated. Two forms of frame fusion are considered: image mosaics and super-resolution. An image mosaic is the alignment of multiple images into a large composition that represents part of a three-dimensional (3D) scene. Super-resolution is a more sophisticated technique that seeks to restore poor-quality video sequences, by modeling and removing the degradations inherent in the imaging process, such as noise, blur, and spatial sampling.

The book begins with a detailed survey of the literature, followed by a chapter on geometric and photometric registration. The next chapter describes a novel algorithm for the efficient matching of image features across multiple views that are related by projective transformations. An automatic method is demonstrated for choosing an optimal viewing transformation, which seeks to minimize projective distortion in the rendered mosaics. The next chapter addresses maximum likelihood and related approaches to super-resolution. The method is compared in detail with Irani and Peleg’s classical super-resolution algorithm [1]. The use of several models of super-resolution, using Bayesian priors, is then discussed, and the performance of such models is investigated empirically. Next, the text describes a novel approach to super-resolution, using sub-space models for super-resolution, both in maximum likelihood and maximum a posterior estimation. The book concludes with several possible avenues of future research.

A key element in this thesis is the assumption of a completely uncalibrated camera. No prior knowledge of the camera parameters, its motion, optics, or photometric characteristics is assumed.

This book, part of Springer’s distinguished dissertations series, has no indication of the author’s school or dissertation date, or whether the book is a revised version of the dissertation. The date must be 2001 or so, based on the latest reference cited, and the school is probably Oxford, since Andrew Zisserman was the thesis advisor.

The material is clearly presented, and the power of the methods is illustrated with many real image sequence examples, including four pages of color plates. The book provides a thorough bibliography of references through to 2001.