Long on measurement and short on consequence, this study searches for a relationship between user engagement and reading comprehension. Finding meager links, the authors report that the study adds “further evidence that the relationship between engagement and comprehension is complex and mediated.”

The research is a tour de force, however, in theory, measurement, and analysis. At least seven instruments were administered to study participants in a two-hour session that allowed 30 minutes for the experimental treatment. The treatment was the on-screen reading of three items (journal article, popular article, and website) on a selected topic. The measures were of user engagement (four subscales); assessments of text comprehension (both microstructural and macrostructural elements); a standardized reading test; an assessment of reading motivation; and self-reports of topical interest (pre and post), motivation, and knowledge gained. Also, eight experimental conditions were provided by the choice of two reading topics and four presentation styles. All of this with a sample size of 41 students.

With lots of data, the researchers did unearth a few relationships, even if the primary goal of linking user engagement to reading comprehension was largely missed. An inconsistent association was found with text comprehension in one of the four subscales of engagement. All participants reported increased knowledge from the readings, and “non-engaged” participants showed a decline in topical interest in pre/post measures.

The authors mention the small sample size at least seven times as affecting their analysis, clarifying that “our findings should be treated as exploratory.”