Publication Date
Spring 2025
Degree Type
Master's Project
Degree Name
Master of Science in Computer Science (MSCS)
Department
Computer Science
First Advisor
Faranak Abri
Second Advisor
William Andreopoulos
Third Advisor
Sayma Akther
Keywords
Fine-tuning, LLM, phishing, prompt engineering, scenario generation, social engineering.
Abstract
Social engineering is found in a strong majority of cyberattacks today, as it is a powerful manipulation tactic that does not require the technical skills of hacking. Calculated social engineers utilize simple communication to deceive and exploit their victims, all by capitalizing on the vulnerabilities of human nature: trust and fear. When successful, this inconspicuous technique can lead to millions of dollars in losses. Social engineering is not a one-dimensional technique; criminals often leverage a combination of strategies to craft a robust yet subtle attack. In addition, offenders are continually evolving their methods in efforts to surpass preventive measures. A common utility to defend against social engineering attacks is detection-based software. Security awareness, however, is a valuable approach that is often eclipsed by automated tech solutions. Awareness establishes a strong first line of defense against these ever-changing attacks. This study utilizes three data-supplemented large language models to generate custom social engineering scenarios with the goal of supporting strong example-driven security awareness programs. The performances of BERT, GPT-3.5, and Llama 3.1 are comparatively analyzed, with Llama 3.1 producing the highest quality scenarios based on a series of metrics, including LLM-as-a-judge.
Recommended Citation
Webb, Jade, "Social Engineering Scenario Generation for Awareness-Based Attack Resilience" (2025). Master's Projects. 1470.
DOI: https://doi.org/10.31979/etd.56jj-yhah
https://scholarworks.sjsu.edu/etd_projects/1470
