TY - JOUR
T1 - A Crowdsourcing-driven AI Model Design Framework to Public Health Policy-Adherence Assessment
AU - Zhang, Yang
AU - Zong, Ruohan
AU - Shang, Lanyu
AU - Wang, Dong
N1 - Publisher Copyright:
© 2013 IEEE.
PY - 2024
Y1 - 2024
N2 - This paper focuses on a public health policy-adherence assessment (PHPA) application that aims to automatically assess people's public health policy adherence during emergent global health crisis events (e.g., COVID-19, MonkeyPox) by leveraging massive public health policy adherence imagery data from the social media. In particular, we study an optimal AI model design problem in the PHPA application, where the goal is to leverage the crowdsourced human intelligence to accurately identify the optimal AI model design (i.e., network architecture and hyperparameter configuration combination) without the need of AI experts. However, two critical challenges exist in our problem: 1) it is challenging to effectively optimize the AI model design given the interdependence between network architecture and hyperparameter configuration; 2) it is non-trivial to leverage the human intelligence queried from ordinary crowd workers to identify the optimal AI model design in the PHPA application. To address these challenges, we develop CrowdDesign, a subjective logic-driven human-AI collaborative learning framework that explores the complementary strength of AI and human intelligence to jointly identify the optimal network architecture and hyperparameter configuration of an AI model in the PHPA application. The experimental results from two real-world PHPA applications demonstrate that CrowdDesign consistently outperforms the state-of-the-art baseline methods by achieving the best PHPA performance.
AB - This paper focuses on a public health policy-adherence assessment (PHPA) application that aims to automatically assess people's public health policy adherence during emergent global health crisis events (e.g., COVID-19, MonkeyPox) by leveraging massive public health policy adherence imagery data from the social media. In particular, we study an optimal AI model design problem in the PHPA application, where the goal is to leverage the crowdsourced human intelligence to accurately identify the optimal AI model design (i.e., network architecture and hyperparameter configuration combination) without the need of AI experts. However, two critical challenges exist in our problem: 1) it is challenging to effectively optimize the AI model design given the interdependence between network architecture and hyperparameter configuration; 2) it is non-trivial to leverage the human intelligence queried from ordinary crowd workers to identify the optimal AI model design in the PHPA application. To address these challenges, we develop CrowdDesign, a subjective logic-driven human-AI collaborative learning framework that explores the complementary strength of AI and human intelligence to jointly identify the optimal network architecture and hyperparameter configuration of an AI model in the PHPA application. The experimental results from two real-world PHPA applications demonstrate that CrowdDesign consistently outperforms the state-of-the-art baseline methods by achieving the best PHPA performance.
UR - http://www.scopus.com/inward/record.url?scp=85209918560&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85209918560&partnerID=8YFLogxK
U2 - 10.1109/TETC.2024.3496835
DO - 10.1109/TETC.2024.3496835
M3 - Article
AN - SCOPUS:85209918560
SN - 2168-6750
JO - IEEE Transactions on Emerging Topics in Computing
JF - IEEE Transactions on Emerging Topics in Computing
ER -