Publication Date

Fall 2023

Degree Type

Master's Project

Degree Name

Master of Science in Computer Science (MSCS)


Computer Science

First Advisor

Fabio Di Troia

Second Advisor

Thomas Austin

Third Advisor

Venkat J Thodima


Malware detection, Machine learning algorithms, Convolutional Neural Networks, AlexNet, LeNet, VGGNet


In recent years, deep learning has emerged as a powerful tool for image classification tasks. However, the performance of individual deep learning models can be limited by their architecture and training data. In this project, various Convolutional Neural Network (CNN) architectures are proposed to train the malware data for feature extraction for various color coordinates such as L, CMYK, RGB, RGBA, and YCbCr. Different optimization techniques like Stochastic Gradient Descent, Root Mean Square Propagation, Ada Delta, Adam, and Adaptive Gradient are used to minimize errors in the trained data, leading to enhanced accuracy. The proposed ensemble deep learning model for image classification, combining strengths from popular CNN architectures like LeNet, AlexNet, VGG, ResNet, and GoogleNet, is utilized for feature extraction. This model employs a weighted average of predictions from each model to make the final classification decision. The performance of the proposed model is evaluated using several benchmark datasets, and the experimental results demonstrate that the ensemble model achieves higher accuracy and robustness compared to individual models and other state-of-the-art ensemble methods. Ablation studies were conducted to analyze the contribution of each model to the ensemble performance. The results reveal that each model contributes differently, and the combination of all models achieves the best performance. The proposed ensemble deep learning model, incorporating AlexNet, LeNet, VGG, ResNet, and GoogleNet architectures, attains state-of-the-art performance across several benchmark datasets. This versatile model can be applied to various image classification tasks, including object recognition, scene understanding, and medical image analysis.

Available for download on Friday, December 20, 2024