This book focuses on the important and interesting topic of automatic usability evaluation, which has cost-saving potential for designing, deploying, and maintaining applications. This topic, especially automation, is even more relevant in the multi-target world of applications because current applications should work on several kinds of devices (not just on one) with various form factors and interaction paradigms. The proposed automatic usability evaluation predicts user performance (time to complete a task) and user errors. For this purpose, it employs models of cognition and human error, and models of user interfaces to simulate user behavior while using the evaluated application.
Usability evaluations mainly consider how well users can learn and use an application/product to achieve their goals. Most existing books on usability evaluation discuss qualitative user studies based on widely used heuristics or principles (good practices). This book takes a different approach in its focus on automatic usability evaluation. Automatability is achieved via computational cognitive modeling, the basis of the presented approach.
The book consists of three parts, references, and an index. The first part discusses related work, mainly different models of human cognition, human error, and user interfaces. It is well written and provides a nice overview of the topic. However, I missed a discussion of error severity, since not all errors are created equal, that is, some can lead to task failure, others only prolong the task completion time (but the user can still successfully complete the task).
Parts 2 and 3 evaluate different models for predicting user performance and errors. The author achieves promising results and shows different aspects that impact performance and the probability of error occurrence. I appreciate that the author incorporated prior knowledge into the model, though only in the form of objective knowledge via an external knowledge base.
The author convincingly shows the feasibility of predicting user performance and error, and how to employ the information contained in user interface models. The main limitations of the work--which the author identifies and discusses--are in the practicality of such predictions and their possible application, for example, significant differences in user performance often only differ in hundreds of milliseconds.
Furthermore, the predicted errors are perhaps on too fine a granularity. For example, I would have expected the author to first predict the task completion and then proceed to a more fine-grained approach; however, this was not the case. This makes the approach more suitable for safety-critical applications (such as software for power plants) or for business applications where the prevention of errors (even noncritical ones) can lead to significant cost and time savings.
The book presents research work on an important practical problem. I like the connection between a model-based approach and practical problems, and appreciate its evaluation on real-world applications. I would recommend it mostly to researchers in this area. It is an excellent resource for both beginning and advanced researchers.
There are several possible extensions of this work, including a user model that incorporates an individual user’s prior knowledge, skill, and information (computer) literacy, all of which impact usability and the probability of error. Additionally, usability also includes learnability and other aspects not researched here (for example, could the model predict the time necessary to learn to use the application correctly?). Moreover, the author shows (in line with previous research) that usability and user performance have only a small direct impact on perceived user experience (subjective ratings of the interface). It would be interesting to explore this in more depth and show the possible tradeoffs between these important aspects of a user interface. Readers can explore these remaining open research questions, but should consider other resources besides those listed in the book, as many are not recent.
