Publication Date

Spring 2023

Degree Type

Master's Project

Degree Name

Master of Science (MS)

Department

Computer Science

First Advisor

Navrati Saxena

Second Advisor

Robert Chun

Third Advisor

Abhishek Roy

Keywords

facial inpainting, CNNs

Abstract

The use of face masks due to the covid-19 pandemic has made surveillance of people very difficult. Since a mask covers most of the facial components, security cameras are rendered of little to no use in the identification of criminals. In order to realize what a face looks like behind a mask, we have to construct the facial features in the masked region. On a higher level, this falls under the field of image inpainting, i.e. filling missing regions of images or correcting irregularities in images. Current research on image inpainting shows promising results on images that have missing/incorrect patches or have manually blackened/whitened pixels. However, they have not been designed to handle the specific case of facial feature generation beneath a mask. The challenge lies in the fact that the mask becomes a bunch of pixels that are on the face in different natural colours, types(surgical, cloth, etc) and shapes(N95, N90, etc). In this paper I provide a custom U-Net based Convolutional Neural Network to regenerate the face underneath the mask. I also create a synthetic dataset of artificially masked human faces with various types of masks. Initial experiments demonstrated superior performance in terms of both quality, diversity over different types of facial coverings/masks and free-form image completion with little to no error.

Download

Share

COinS