Tripartite Intelligence: Synergizing Deep Neural Network, Large Language Model, and Human Intelligence for Public Health Misinformation Detection (Archival Full Paper)

Yang Zhang, Ruohan Zong, Lanyu Shang, Zhenrui Yue, Huimin Zeng, Yifan Liu, Dong Wang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The threat of rapidly spreading health misinformation through social media during crises like COVID-19 emphasizes the importance of addressing both clear falsehoods and complex misinformation, including conspiracy theories and subtle distortions. This paper designs a novel tripartite collective intelligence approach that integrates deep neural networks (DNNs), large language models (LLMs), and crowdsourced human intelligence (HI) to collaboratively detect complex forms of public health misinformation on social media. Our design is inspired by the collaborative strengths of DNNs, LLMs, and HI, which complement each other. We observe that DNNs efficiently handle large datasets for initial misinformation screening but struggle with complex content and rely on high-quality training data. LLMs enhance misinformation detection with improved language understanding but may sometimes provide eloquent yet factually incorrect explanations, risking misinformation mislabeling. HI provides critical thinking and ethical judgment superior to DNNs and LLMs but is slower and more costly in misinformation detection. In particular, we develop TriIntel , a tripartite collaborative intelligence framework that leverages the collective intelligence of DNNs, LLMs, and HI to tackle the public health information detection problem under a novel few-shot and uncertainty-aware maximum likelihood estimation framework. Evaluation results on a real-world public health misinformation detection application related to COVID-19 show that TriIntel outperforms representative DNNs, LLMs, and human-AI collaboration baselines in accurately detecting public health misinformation under a diverse set of evaluation scenarios.

Original languageEnglish (US)
Title of host publicationCI 2024
Subtitle of host publicationProceedings of the ACM Collective Intelligence Conference
PublisherAssociation for Computing Machinery
Pages63-75
Number of pages13
ISBN (Electronic)9798400701139
DOIs
StatePublished - Jun 27 2024
Event2024 ACM Collective Intelligence Conference, CI 2024 - Boston, United States
Duration: Jun 27 2024Jun 28 2024

Publication series

NameCI 2024: Proceedings of the ACM Collective Intelligence Conference

Conference

Conference2024 ACM Collective Intelligence Conference, CI 2024
Country/TerritoryUnited States
CityBoston
Period6/27/246/28/24

Keywords

  • Collective Intelligence
  • Human-AI Collaboration
  • Large Language Model
  • Misinformation

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Networks and Communications
  • Computer Science Applications
  • Software

Fingerprint

Dive into the research topics of 'Tripartite Intelligence: Synergizing Deep Neural Network, Large Language Model, and Human Intelligence for Public Health Misinformation Detection (Archival Full Paper)'. Together they form a unique fingerprint.

Cite this