Enhancing Cognitive Assessment through Multimodal Sensing: A Case Study Using the Block Design Test

Many cognitive assessments are limited by their reliance on relatively sparse measures of performance, like per-item accuracy or reaction time. Capturing more detailed behavioral measurements from cognitive assessments will enhance their utility in many settings, from individual clinical evaluations to large-scale research studies. We demonstrate the feasibility of combining scene and gaze cameras with supervised learning algorithms to automatically measure key behaviors on the block design test, a widely used test of visuospatial cognitive ability. We also discuss how this block-design measurement system could enhance the assessment of many critical cognitive and meta-cognitive functions such as attention, planning, progress monitoring, and strategy selection.