Publication Date
Fall 2025
Degree Type
Master's Project
Degree Name
Master of Science in Computer Science (MSCS)
Department
Computer Science
First Advisor
Mark Stamp
Second Advisor
Fabio Di Troia
Third Advisor
Navrati Saxena
Keywords
large language models, malicious URL detection, zero-shot learning, model inconsistency
Abstract
In this paper, we consider the effectiveness of specific Large Language Models (LLMs) for detecting malicious URL in a phishing URL dataset. Specifically, we investigate whether multiple classification attempts made with an LLM can improve the classification accuracy. This research aims to analyze the inconsistency in LLMs when used to classify malicious and non-malicious URLs. We find that the accuracy of most LLMs is little better than a random classifier, and multiple classification attempts only provide a marginal improvement.
Recommended Citation
Lotwala, Jhanvi, "On the Effectiveness of Large Language Models for Classifying Malicious URLs" (2025). Master's Projects. 1612.
DOI: https://doi.org/10.31979/etd.2sgx-ekjt
https://scholarworks.sjsu.edu/etd_projects/1612
