Verhealth Vetting medical voice applications through policy enforcement

Faysal Hossain Shezan, Hang Hu, Gang Wang, Yuan Tian

Research output: Contribution to journalArticlepeer-review

Abstract

Healthcare applications on Voice Personal Assistant System (e.g., Amazon Alexa), have shown a great promise to deliver personalized health services via a conversational interface. However, concerns are also raised about privacy, safety, and service quality. In this paper, we propose VerHealth, to systematically assess health-related applications on Alexa for how well they comply with existing privacy and safety policies. VerHealth contains a static module and a dynamic module based on machine learning that can trigger and detect violation behaviors hidden deep in the interaction threads. We use VerHealth to analyze 813 health-related applications on Alexa by sending over 855,000 probing questions and analyzing 863,000 responses. We also consult with three medical school students (domain experts) to confirm and assess the potential violations. We show that violations are quite common, e.g., 86.36% of them miss disclaimers when providing medical information; 30.23% of them store user physical or mental health data without approval. Domain experts believe that the applications' medical suggestions are often factually-correct but are of poor relevance, and applications should have asked more questions before providing suggestions for over half of the cases. Finally, we use our results to discuss possible directions for improvements.

Original languageEnglish (US)
Article number3432233
JournalProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Volume4
Issue number4
DOIs
StatePublished - Dec 17 2020

Keywords

  • Alexa
  • Google-Home
  • Medical-voice-Applications
  • Skill
  • dynamic-Analysis
  • policy-enforcement

ASJC Scopus subject areas

  • Human-Computer Interaction
  • Hardware and Architecture
  • Computer Networks and Communications

Fingerprint

Dive into the research topics of 'Verhealth Vetting medical voice applications through policy enforcement'. Together they form a unique fingerprint.

Cite this