Publication Date

Fall 2023

Degree Type

Master's Project

Degree Name

Master of Science in Computer Science (MSCS)

Department

Computer Science

First Advisor

William Andreopoulos

Second Advisor

Navrati Saxena

Third Advisor

Robert Chun

Keywords

Disaster Detection, Social Media Analysis, Large Language Models, Machine Learning, Comparative Analysis, Twitter, Emergency Response, Linguistic Nuances

Abstract

Disaster Detection using Twitter content is critical for emergency response, but accurately identifying relevant tweets remains challenging due to nuances, informal language, and emotional expressions. This paper presents a comparative analysis between traditional Machine Learning models, Deep Learning models and Large Language Models (LLM) for classifying disaster vs. non-disaster tweets. While existing works have applied pattern recognition and dataset-specific learning, LLMs with their deeper understanding of linguistics and semantics can potentially handle the complexities of tweets more effectively. This study leverages LLMs including Llama2, Mistral, and Falcon, Open AI GPT 3.5, hypothesizing their superior contextual comprehension will excel in tweets laden with ambiguities and errors. A comparative evaluation highlights instances where LLMs provide enhanced accuracy and language interpretation over prevailing approaches like CNN-LSTM networks, SVM, and Random Forest classifiers. Results demonstrate LLMs significantly improving detection of nuanced language, handling linguistic variabilities, and adapting to evolving language better. These advantages are critical for reliable disaster detection on Twitter during time-sensitive emergency events.

Download

Share

COinS