TY - JOUR
T1 - Exploring artificial intelligence-powered virtual assistants to understand their potential to support older adults’ search needs
AU - Langston, Emily M.
AU - Hattakitjamroen, Varitnan
AU - Hernandez, Mario
AU - Lee, Hye Soo
AU - Mason, Hannah
AU - Louis-Charles, Willencia
AU - Charness, Neil
AU - Czaja, Sara J.
AU - Rogers, Wendy A.
AU - Sharit, Joseph
AU - Boot, Walter R.
N1 - This research was supported in part by the National Institutes of Health (National Institute on Aging) grant [ P01AG073090 ] under the auspices of the Center for Research and Education on Aging and Technology Enhancement (CREATE ; www.create-center.org ).
PY - 2025/6
Y1 - 2025/6
N2 - Objective: We investigated the accuracy and amount of information provided by artificial intelligence (AI)-powered virtual assistants in response to queries relevant to aging adults in the domains of Medicare, long-term care insurance, and resource access. Background: Older adults are faced with complex decisions and must gather and integrate information from diverse sources to help support these decisions (e.g., across various websites and online resources). Information-seeking, integration, and decision-making are cognitively demanding and can be impacted by age-related cognitive changes. Virtual assistants powered by AI have the potential to provide older adults with easy access to information and answers to their queries. However, it is unclear how accurate this information and these answers might be. Method: Alexa, Google Assistant, Bard, and ChatGPT-4 were queried. Coders assessed the accuracy of these responses, and the amount of supplemental information provided as a measure of response complexity. Results: Overall, Large Language Model (LLM)-based virtual assistants (Bard, ChatGPT-4) responded more accurately than non-LLM assistants (e.g., 6 % inaccurate responses for Bard vs. 60 % for Alexa) and provided substantially more supplemental information (79 % of responses with high supplemental information for Bard and 37 % for Chat-GPT, vs. 20 % or less for others). We note, however, that responses can vary over time. Conclusion: Based on their ability to provide largely accurate responses, LLMs may be helpful tools for older adults seeking information related to health, insurance, and available resources. However, the potential for error, high response complexity, and response variability should be considered. Application: LLM-based virtual assistants may be a helpful tool for older adults seeking information to support health and financial decisions.
AB - Objective: We investigated the accuracy and amount of information provided by artificial intelligence (AI)-powered virtual assistants in response to queries relevant to aging adults in the domains of Medicare, long-term care insurance, and resource access. Background: Older adults are faced with complex decisions and must gather and integrate information from diverse sources to help support these decisions (e.g., across various websites and online resources). Information-seeking, integration, and decision-making are cognitively demanding and can be impacted by age-related cognitive changes. Virtual assistants powered by AI have the potential to provide older adults with easy access to information and answers to their queries. However, it is unclear how accurate this information and these answers might be. Method: Alexa, Google Assistant, Bard, and ChatGPT-4 were queried. Coders assessed the accuracy of these responses, and the amount of supplemental information provided as a measure of response complexity. Results: Overall, Large Language Model (LLM)-based virtual assistants (Bard, ChatGPT-4) responded more accurately than non-LLM assistants (e.g., 6 % inaccurate responses for Bard vs. 60 % for Alexa) and provided substantially more supplemental information (79 % of responses with high supplemental information for Bard and 37 % for Chat-GPT, vs. 20 % or less for others). We note, however, that responses can vary over time. Conclusion: Based on their ability to provide largely accurate responses, LLMs may be helpful tools for older adults seeking information related to health, insurance, and available resources. However, the potential for error, high response complexity, and response variability should be considered. Application: LLM-based virtual assistants may be a helpful tool for older adults seeking information to support health and financial decisions.
KW - Aging
KW - Artificial intelligence
KW - Decision-making
KW - Health
KW - Information search
UR - http://www.scopus.com/inward/record.url?scp=85215836689&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85215836689&partnerID=8YFLogxK
U2 - 10.1016/j.hfh.2025.100092
DO - 10.1016/j.hfh.2025.100092
M3 - Article
AN - SCOPUS:85215836689
SN - 2772-5014
VL - 7
JO - Human Factors in Healthcare
JF - Human Factors in Healthcare
M1 - 100092
ER -