Publication Date

Spring 2026

Degree Type

Master's Project

Degree Name

Master of Science in Computer Science (MSCS)

Department

Computer Science

First Advisor

Fabio Di Troia

Second Advisor

William Andreopoulos

Third Advisor

Rajat Kabra

Keywords

malware detection, malware family classification, explainable AI, large language models, BODMAS, AV-Agent, static PE analysis

Abstract

The aim of this thesis is to explore how large language models could be combined with conventional malware detection systems, leading to better interpretability while maintaining highly efficient supervised baseline performance. In particular, the motivation for this thesis comes from the recently proposed AV-Agent framework, which proposes malware analysis to be approached as a three-step process, including hierarchical feature generation, feature extraction at the character-level, and two-step reasoning, which involves large language models. Drawing inspiration from the above framework, this thesis seeks to extend the AV-Agent approach to the BODMAS dataset as well as apply it to a more complicated problem in two ways. The primary problem to solve is binary malware classification that determines whether a Windows Portable Executable file is malicious or not along with two-step reasoning using an LLM. The proposed architecture consists of three branches of feature sets that complement each other by extracting dynamic data complementary to the static data such as byte statistics, structural properties, and heuristic PE data. The models are trained independently on each of the branches and then combined to make a malware prediction decision. Phase I reasoning on complex binaries makes use of the confidence score generated for each of the branches, whereas Phase II reasoning on complex binaries is done using the features extracted from each branch using SHAP values. Family extension of the proposed technique follows the same methodology with further steps involving family classifiers, confusing route families, and reasoning packs of candidates. The thesis presents an extensive study regarding the design of an interpretable malware analysis framework that is motivated by the concept of an AV-Agent using the BODMAS dataset. Methods such as hierarchical learning classifier construction, two-stage large language models prediction, and malware family categorization have been used to increase the accuracy of the prediction and its interpretability. This document outlines the complete procedure for undertaking the research starting from the problem formulation until the data analysis and methodology steps.

Download

Available for download on Saturday, May 22, 2027

Share

COinS