Chapman reports on an effort to build an integrated artificial intelligence system that includes aspects of vision, computational linguistics, and activity theory (how an agent interacts with its environment). This book is derived from his Ph.D. dissertation. The first chapter provides a summary of the problem domain, a computer program playing a video game using visual analysis and outside advice. Like most of the individual chapters but more globally, this chapter provides a guide to reading the book, with suggestions of what to skip and what is important for understanding.

Chapman’s “concrete-situated” approach is based on the idea of performing activities in the world. It focuses on routine activities that depend on the current situation, interact with the environment (which also influences the agent), and are dynamic and improvisational. This approach is described in chapter 2, which also presents the basic ideas of the deictic representation, which describes things in terms of their relation to the agent.

Chapter 3 discusses the biological motivation for this implementation and introduces the architecture for the central control system and the vision processing. The goal of exploring biologically plausible implementations of activity led to a connectionist architecture based on simulated digital circuits for the implementation of the central control system. The different components are described in detail, beginning with the video game task, called Amazon, in chapter 4.

Chapter 5 discusses the simple natural language interface that is used to give advice to the program on how to play the game and explains how the program uses that advice to choose the appropriate action. The natural language is limited and directed only at giving advice about the game. The usual issues for language understanding, such as syntax and reference, are ignored.

With input to the vision system from the descriptions used for display, the author has no need to build a complete vision system for real images. This allows him to avoid many of the difficult issues encountered by other computer vision researchers, such as how to extract and recognize the basic objects in a scene. The vision system is based on a set of intermediate-level visual operators (which interface between image-level representations and compact, symbolic representations) to perform basic operations. By combining these operators, the system performs visual tasks required for the video game problem, such as search. These general-purpose visual operators are similar to the processing that is done in many other vision systems, but their combination is specific to this task.

Chapter 7 describes the simulated digital circuit model used to model the overall system, which is based on the biological connectionist model. The next chapter explores the operation of the entire system, with illustrations of how the visual operators are applied, how advice is given and used, and how the program reacts to different problems in the course of a game.

The final chapter evaluates the results of the implementation and discusses areas of further study. Several pages discuss how this work should be evaluated since, according to the author, unlike mathematics it has no theorems, unlike pure science it has no tested theories, unlike engineering it has no superior technologies, and unlike philosophy it has no rigorous arguments. Chapman argues that artificial intelligence is dominated by approaches (new ways to look at a problem), and that is the proper way to evaluate this work. In the evaluation of the connectionist architecture, he observes that almost any implementation would work, since choosing what to do is not difficult, but perception was harder than expected. This conclusion mirrors that of many others who have looked at computer vision over the past two decades.

Because of the variety of its contributions, it is difficult to place this book in any one category. It is a readable discussion of the application of the connectionist approach to a particular problem and is more important for its contributions to modeling intelligence than for its contributions to computer vision. This emphasis is reflected in the bibliography, which is devoted more to models of intelligence than to computer vision.