Single Cross-domain Semantic Guidance Network for Multimodal Unsupervised Image Translation
Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Multimodal image-to-image translation has received great attention due to its flexibility and practicality. The existing methods lack the generality of effective style representation, and cannot capture different levels of stylistic semantic information from cross-domain images. Besides, they ignore the parallelism for cross-domain image generation, and their generator can only be responsible for specific domains. To address these issues, we propose a novel Single Cross-domain Semantic Guidance Network (SCSG-Net) for coarse-to-fine semantically controllable multimodal image translation. Images from different domains are mapped to a unified visual semantic latent space by a dual sparse feature pyramid encoder, and then the generative module generates the result images by extracting semantic style representation from the input images in a self-supervised manner guided by adaptive discrimination. Especially, our SCSG-Net meets the needs of users in different styles as well as diverse scenarios. Extensive experiments on different benchmark datasets show that our method can outperform other state-of-the-art methods both quantitatively and qualitatively.
Department of Natural Resources of Guangdong Province
Multimodal image translation, Semantic guidance, Unsupervised learning
Industrial and Systems Engineering
Jiaying Lan, Lianglun Cheng, Guoheng Huang, Chi Man Pun, Xiaochen Yuan, Shangyu Lai, Hong Rui Liu, and Wing Kuen Ling. "Single Cross-domain Semantic Guidance Network for Multimodal Unsupervised Image Translation" Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (2023): 165-177. https://doi.org/10.1007/978-3-031-27077-2_13