Providing Real-World Benchmarks for Super-Resolving Fluorescence Microscope Imagery Using Generative Adversarial Networks

Publication Date

1-1-2024

Document Type

Conference Proceeding

Publication Title

Proceedings - 2024 IEEE Conference on Artificial Intelligence, CAI 2024

DOI

10.1109/CAI59869.2024.00206

First Page

1154

Last Page

1161

Abstract

Super-resolving and de-noising video-rate acquisition microscope imagery can eliminate the need for practitioners to choose between image acquisition speed and image quality. This task provides a convenient use case for Generative Adversarial Networks (GANs), which have demonstrated impressive pixel-wise reconstruction metrics in microscope image super-resolution tasks. However, the benchmarks which report these metrics typically do so on low-resolution images generated through simple synthetic degradations, which fail to recreate the natural distortions imparted by experimental image acquisition. Because most reported synthetic data generation pipelines rely on the bicubic downsampling of a few high-resolution images, networks trained to correct for this distortion eventually fail downstream when required to super-resolve images containing natural distortions. The literature therefore provides us with an unreliable assessment of GANs' ability to super-resolve microscope imagery in the field. In this work, we present one of the few examples of GANs successfully super-resolving a large cache of experimentally gathered microscope imagery. For our main result, we demonstrate a reliable baseline for the super-resolution task using GAN, in which we obtain a peak-signal-to-noise ratio (PSNR) of 29.21 and a structural similarity index (SSIM) of 0.845 by using all available image pairs and averaging across all sub-datasets. To demonstrate robustness on this task, we present the model with a blind super-resolution task, in which it achieves a PSNR of 25.75 and SSIM of 0.676 after averaging across all sub-datasets. To affirm our results as a reliable baseline, we demonstrate that GANs can fail in the video-rate super-resolution task even when trained on higher-order synthetic degradation pipelines. We confirm this effect by training our model on purely synthetic data, using the pipeline mentioned above, and testing it on a single sub-dataset. In doing so, we observe a -0.06 loss in SSIM and -0.75 loss in PSNR, accompanied by significant quality degradation of the reconstructed images in the form of severe distortion and artifact generation.

Keywords

bicubic downsampling, GAN, PSNR, SSIM, super-resolution

Department

Mathematics and Statistics

Link to Full Text

Share

COinS