Publication Date
Fall 2020
Degree Type
Master's Project
Degree Name
Master of Science (MS)
Department
Computer Science
First Advisor
Mark Stamp
Second Advisor
Rula Khayrallah
Third Advisor
Thomas Austin
Keywords
Neuro-Evolution of Augmenting Topologies, Malware Classification
Abstract
Current malware detection software often relies on machine learning, which is seen as an improvement over signature-based techniques. Problems with a machine learning based approach can arise when malware writers modify their code with the intent to evade detection. This leads to a cat and mouse situation where new models must constantly be trained to detect new malware variants. In this research, we experiment with genetic algorithms as a means of evolving machine learning models to detect malware. Genetic algorithms, which simulate natural selection, provide a way for models to adapt to continuous changes in a malware families, and thereby improve detection rates. Specifically, we use the Neuro-Evolution of Augmenting Topologies
(NEAT) algorithm to optimize machine learning classifiers based on decision trees and neural networks. We compare the performance of our NEAT approach to standard models, including random forest and support vector machines.
Recommended Citation
Do, Jason, "A NEAT Approach to Malware Classification" (2020). Master's Projects. 973.
DOI: https://doi.org/10.31979/etd.6kjs-n786
https://scholarworks.sjsu.edu/etd_projects/973
