Multi-text multi-modal reading processes and comprehension

Learners frequently access multiple, illustrated texts on a single topic, but this kind of multi-text, multi-modal comprehension has been rarely studied. Undergraduates were presented with two sets of biology materials on shared topics, including both texts and captioned illustrations. Process data, including think-alouds and screen recordings of participants' attendance to different foci, were recorded. Both intra-textual (e.g., T1, T1) and inter-textual (e.g., T1, T2) navigation sequences and switches from texts to diagrams (e.g, T1, D1) were recorded and examined in relation to comprehension performance. Patterns in participants’ navigation across texts and illustrations were identified. Students who were higher-performing in the sample exhibited more intra-textual switches, suggestive of forming a page-specific mental model. For one text, inter-textual text-to-diagram switches were negatively associated with high-level strategies, and thus negatively associated with performance; inter-textual text-to-diagram switches might have indicated confusion. Participants appeared to be in the early stages of learning a complex set of MTMM comprehension skills.