Publication Date

Spring 2023

Degree Type

Master's Project

Degree Name

Master of Science (MS)

Department

Computer Science

First Advisor

Ching-seh Wu

Second Advisor

William Andreopoulos

Third Advisor

Ibrahim Khoury

Keywords

Keywords— Sign Language, Neural networks, Machine learning, American Sign Language, Ensemble model

Abstract

Sign Language is a visual language used by millions of people around the world. American Sign Language (ASL) is one of the most popular sign languages and the third most popular language in the United States. Automatic recognition of ASL signs can help bridge the communication gap between deaf and hearing individuals. In this project, we explore the use of deep learning models for ASL sign recognition, using the MNIST dataset as a benchmark. We preprocessed the data by reshaping the images to the input layer size of the models and normalized the pixel values. We evaluated five popular deep-learning models for image classification: ResNet50, LeNet, AlexNet, VGG16, and DenseNet121. We trained and tested each model on the MNIST dataset, using metrics such as accuracy, mean absolute error (MAE), precision, and recall to evaluate their performance. We also computed the mean squared error (MSE) and confusion matrix to analyze the model's error patterns. Next, we explored ensemble learning techniques to further improve the accuracy of the ASL recognition model. We selected ResNet50, AlexNet, and LeNet as the three best-performing models, and tested two ensemble methods: concatenation and stacking. We found that stacking gave promising accuracy, outperforming concatenation by a significant margin. In conclusion, our study demonstrates the effectiveness of deep learning models for ASL recognition, and the potential of ensemble learning techniques to further improve accuracy from 97% recorded in existing models to 99% using our ensemble model. Our findings could have practical applications in the development of assistive technologies for individuals in need.

Download

Share

COinS