Conversational agents in mHealth: use patterns, challenges, and design opportunities for individuals with visual impairments

Background: Individuals with visual impairments often encounter significant accessibility challenges in using mobile health (mHealth) technologies such as apps and wearable sensor devices. In this context, conversational agents (CAs) hold potential as transformative tools for facilitating their interaction with mHealth. However, no prior research has investigated their needs and challenges in mHealth’s conversational agents. To address this gap, this study investigated the use patterns, challenges, and design opportunities for individuals with visual impairments. Methods: This study involved an online survey of 87 adults with low vision or blindness, aiming to understand their usage patterns, challenges, and needs concerning CAs in mHealth. Results: Our analysis identified seven major challenges related to visual impairments, including dependency on visual feedback, difficulties in learning CA operation, ambiguous operating status of CAs, inaccessible content in user interfaces, a preference for multimodal interaction, suboptimal audio quality, and issues with manual data entry. Additionally, challenges unrelated to visual impairment were noted, such as errors in speech comprehension, inaccuracies in speech recognition, limitations in dialogue continuity, insufficient depth of provided information, inconsistencies in information accuracy, and misinterpretation of homophones. Furthermore, the study uncovered participants’ expectations regarding future CAs, emphasizing their potential role in enhancing physical activity and wellness. Conclusion: Based on our research results, we proposed a series of features and functionalities designed to enhance accessibility and usability in this field. Our recommendations are expected to establish a foundation for future advancements in conversational technology, emphasizing the principles of inclusivity and user-centered design.