The book deals with the simulation of face-to-face communication and the construction of virtual conversational agents. It focuses on the interaction of verbal and non-verbal communicative acts and on the mediation of personality and emotion. Virtual conversational agents can serve as interfaces for human computer users, especially to information or tutorial systems. Interaction with these interfaces should be simple and natural; therefore agents do not only use speech, but also gesture and mimicking actions. The agents discussed are displayed as animated characters or heads - this book is not about physically embodied agents (robots), distributed artificial intelligence or communication between autonomous agents. The book contains 13 articles and is divided into three parts: the introduction, a discussion of systems and a discussion of evaluation. Cassell’s introduction is a very readable overview of multimodal face-to-face communication. Non-verbal communicative acts, like emblematic, propositional and deictic gestures, are distinguished. Non-verbal communicative acts support verbal acts. Illustrative examples show how speech, gesture, gaze, and so on interact.

In Part 2, eight articles present implementations of conversational agent systems. These systems are help, tutorial or general information systems. An agent is animated as a head, a human or a cartoon character in a virtual environment. The agent serves as an interface for the user. The agents act in their environments, and not only speak, but also show the user how certain things work. Rickel and Johnson introduce the agent “Steve,” which acts on a virtual ship and shows how machines on board work. #8220;Cosmo,” presented by Lester et al., is a cartoon character which acts as a tutor, in a learning environment for Internet packet routing. The agent points at things in its environment and refers to these things in its speech. By raising its eyebrows, smiling, and so on, the agent also shows basic emotive behavior in its responses to the user, for example it expresses happiness when the user does something right, or disappointment when the user makes mistakes. André et al. use a different approach: they simulate dialogues between agents. Two agents, “Gerd” and “Matze,” in an agent stage play, comment on a RoboCup football game. By observing the dialogue, the user gets information about what is happening on the field. None of the articles goes into the details of the implementations very deeply, but all articles give bibliographical notes for further information. If existing software is used, then the software is named. The presentations of the systems are accompanied by discussions of the following issues: the meaning of non-verbal communicative acts and their context-sensitivity, the interaction of verbal and non-verbal signals; behavioral primitives; and the coordination of actions in communicative acts, deictic acts, mediation of emotion and personality, naturalness, redundancy and ease of understanding. The four articles of Part 3 reflect the evaluation of conversational agents and user reactions to such agents. It is claimed that communicative acts become easier to understand if they are not only verbal, but also supported by gestures. A user reacts positively to an agent if the agent seems to have a “personality.” Sanders and Scholz develop metrics for the evaluation of conversational agents. They argue that systems have to be evaluated from their conception, starting with “Wizard of Oz” studies, where the agent behavior is simulated by a human.

Development and evaluation of agents are synergistic procedures. All the articles deal with the generation of complex communicative acts, but do not deal with automatic analysis and interpretation of non-verbal acts. Understanding of gaze, gesture, intonation etc. is still a big problem. The book is interesting and inspiring for people who want to use or construct animated conversational agents. It is not a textbook, but can be used in intermediate courses in computer and communication science.