TY - GEN
T1 - Tripartite Intelligence
T2 - 2024 ACM Collective Intelligence Conference, CI 2024
AU - Zhang, Yang
AU - Zong, Ruohan
AU - Shang, Lanyu
AU - Yue, Zhenrui
AU - Zeng, Huimin
AU - Liu, Yifan
AU - Wang, Dong
N1 - Publisher Copyright:
© 2024 ACM.
PY - 2024/6/27
Y1 - 2024/6/27
N2 - The threat of rapidly spreading health misinformation through social media during crises like COVID-19 emphasizes the importance of addressing both clear falsehoods and complex misinformation, including conspiracy theories and subtle distortions. This paper designs a novel tripartite collective intelligence approach that integrates deep neural networks (DNNs), large language models (LLMs), and crowdsourced human intelligence (HI) to collaboratively detect complex forms of public health misinformation on social media. Our design is inspired by the collaborative strengths of DNNs, LLMs, and HI, which complement each other. We observe that DNNs efficiently handle large datasets for initial misinformation screening but struggle with complex content and rely on high-quality training data. LLMs enhance misinformation detection with improved language understanding but may sometimes provide eloquent yet factually incorrect explanations, risking misinformation mislabeling. HI provides critical thinking and ethical judgment superior to DNNs and LLMs but is slower and more costly in misinformation detection. In particular, we develop TriIntel , a tripartite collaborative intelligence framework that leverages the collective intelligence of DNNs, LLMs, and HI to tackle the public health information detection problem under a novel few-shot and uncertainty-aware maximum likelihood estimation framework. Evaluation results on a real-world public health misinformation detection application related to COVID-19 show that TriIntel outperforms representative DNNs, LLMs, and human-AI collaboration baselines in accurately detecting public health misinformation under a diverse set of evaluation scenarios.
AB - The threat of rapidly spreading health misinformation through social media during crises like COVID-19 emphasizes the importance of addressing both clear falsehoods and complex misinformation, including conspiracy theories and subtle distortions. This paper designs a novel tripartite collective intelligence approach that integrates deep neural networks (DNNs), large language models (LLMs), and crowdsourced human intelligence (HI) to collaboratively detect complex forms of public health misinformation on social media. Our design is inspired by the collaborative strengths of DNNs, LLMs, and HI, which complement each other. We observe that DNNs efficiently handle large datasets for initial misinformation screening but struggle with complex content and rely on high-quality training data. LLMs enhance misinformation detection with improved language understanding but may sometimes provide eloquent yet factually incorrect explanations, risking misinformation mislabeling. HI provides critical thinking and ethical judgment superior to DNNs and LLMs but is slower and more costly in misinformation detection. In particular, we develop TriIntel , a tripartite collaborative intelligence framework that leverages the collective intelligence of DNNs, LLMs, and HI to tackle the public health information detection problem under a novel few-shot and uncertainty-aware maximum likelihood estimation framework. Evaluation results on a real-world public health misinformation detection application related to COVID-19 show that TriIntel outperforms representative DNNs, LLMs, and human-AI collaboration baselines in accurately detecting public health misinformation under a diverse set of evaluation scenarios.
KW - Collective Intelligence
KW - Human-AI Collaboration
KW - Large Language Model
KW - Misinformation
UR - http://www.scopus.com/inward/record.url?scp=85203880197&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85203880197&partnerID=8YFLogxK
U2 - 10.1145/3643562.3672613
DO - 10.1145/3643562.3672613
M3 - Conference contribution
AN - SCOPUS:85203880197
T3 - CI 2024: Proceedings of the ACM Collective Intelligence Conference
SP - 63
EP - 75
BT - CI 2024
PB - Association for Computing Machinery
Y2 - 27 June 2024 through 28 June 2024
ER -