Publication Date
5-1-2024
Document Type
Article
Publication Title
Electronics (Switzerland)
Volume
13
Issue
9
DOI
10.3390/electronics13091673
Abstract
Large language models (LLMs) are a compressed repository of a vast corpus of valuable information on which they are trained. Therefore, this work hypothesizes that LLMs such as Llama, Orca, Falcon, and Mistral can be used for misinformation detection by making them cross-check new information with the repository on which they are trained. Accordingly, this paper describes the findings from the investigation of the abilities of LLMs in detecting misinformation on multiple datasets. The results are interpreted using explainable AI techniques such as Local Interpretable Model-Agnostic Explanations (LIME), SHapley Additive exPlanations (SHAP), and Integrated Gradients. The LLMs themselves are also asked to explain their classification. These complementary approaches aid in better understanding the inner workings of misinformation detection using LLMs and lead to conclusions about their effectiveness at the task. The methodology is generic and nothing specific is assumed for any of the LLMs, so the conclusions apply generally. Primarily, when it comes to misinformation detection, the experiments show that the LLMs are limited by the data on which they are trained.
Funding Sponsor
San José State University
Keywords
Captum model interpretability, Cohen’s Kappa score, eXplainable Artificial Intelligence (XAI), greedy decoding, large language models (LLMs), LLM quantization, Matthew’s correlation coefficient (MCC), misinformation containment, natural language processing, zero-shot prompting
Creative Commons License
This work is licensed under a Creative Commons Attribution 4.0 License.
Department
Applied Data Science
Recommended Citation
Vishnu S. Pendyala and Christopher E. Hall. "Explaining Misinformation Detection Using Large Language Models" Electronics (Switzerland) (2024). https://doi.org/10.3390/electronics13091673
