The HUI Analyzer, based on Microsoft’s .NET Compact Framework for handheld devices, is a proof-of-concept tool for automated usability testing. Details of a user interaction, called an actual action sequence, are gathered unobtrusively by a recorder component. A desktop component provides the investigator with the means to specify expected action sequences. The tool can perform comparison checking of expected and actual action sequences. Assertions concerning forms (for example, the percentage of free space), action sequences (for example, the amount of resizing), and the results of comparison checking (for example, the maximum deviation allowed of an actual from an expected action sequence) can all be checked for. The tool also supports hotspot analyses of actual action sequences to reveal how well various graphical user interface (GUI) components are used.

The evaluation study compares working with the HUI Analyzer to conventional formal user testing. Table 5 shows the usability issues revealed by both approaches and Baker et al. conclude that the HUI Analyzer approach compares favorably. The claim is made that working with the HUI Analyzer takes much less time.

The design and reporting of the evaluation study, however, is weak. Sample sizes of eight are more typical of pilot studies. Subjects’ details are missing. The actual time expended to create expected action sequences and assertions is not stated; there is no effort model.

Despite the shortcomings of the evaluation study, the ideas behind the HUI Analyzer are sound and deserving of a wide audience. I strongly recommend this paper to the software testing community.