Computing devices often include sensors to track the body, limbs, hands, and fingers of a user. User interfaces rely on these capabilities to recognize gestures, and considerable energy is being devoted to the development of a core set of gestures. The authors distinguish between two types of gestures. Symbolic gestures rely on the mapping of a sign (“thumbs up,” for example) to an interpretation (“approval”). Manipulation gestures are based on activities grounded in the way humans deal with spatial and physical aspects of objects, like moving an object. Convergence on a core set of manipulation gestures (tapping on an object to select it, or pinching to zoom in and out) seems possible, but less so for abstract gestures.

As an example, the authors discuss gestures that draw a checkmark or an X, with the intendend interpretations of “approval” and “rejection.” These gestures are problematic since the mapping of the signs is ambiguous: on forms, approval or selection of an item is often expressed by marking a box with either a check mark or an X. On touch-based interfaces, selection of an object (which could be a check box) is more intuitive by tapping. Performing the same check mark and X gestures midair (without a touch panel) becomes even worse since the start and end points of a (partial) gesture are not clearly defined.

Our familiarity with the spatial surroundings in which we live, and the way objects behave in the real world, provides us with a good basis for a common set of manipulation gestures. The same, unfortunately, does not seem to apply to more abstract gestures, requiring user effort to learn the mapping between gestures and their intended interpretations. In this relatively short viewpoint paper, the authors emphasize the distinction between gestural manipulations and symbolic gestures. Since manipulation gestures are grounded in the real world, there is a good basis for a common core set of gestures, while the abstract mapping between symbolic gestures and the respective commands may have significant variations in their potential interpretations, and thus can’t be easily captured in such a common core set.