6 June 2023 AttCST: attention improves style transfer via contrastive learning
Lei Zhou, Tao Zhang
Author Affiliations +
Abstract

Arbitrary style transfer has flexible applicability and aims to learn the painting styles of different artists through one model training and re-render to everyday photos. Existing domain-to-domain style transfer methods based on generative adversarial networks achieve good results in image generation quality but need more flexibility in various style transfer tasks. In the image-to-image style transfer method, focusing on the fusion of content image and style image can learn local detail texture very well. Still, there are some defects in the preservation of the content structure. To alleviate the above two problems, we leverage both advantages, using attention improves style transfer via contrastive learning. Specifically, the attention mechanism is used to fuse the multi-scale features of the content and style images. The previous shallow features are fully utilized to supplement the texture information lost by the deep features. Then the currently learned style image is distinguished from other style images by contrastive learning modules, thereby enhancing the ability of the network to learn style images. We also derive a local loss based on the transferred blocks to improve the visual quality of the generated images. We conduct extensive qualitative and quantitative analyses, the results of which demonstrate the effectiveness of our approach.

© 2023 SPIE and IS&T
Lei Zhou and Tao Zhang "AttCST: attention improves style transfer via contrastive learning," Journal of Electronic Imaging 32(3), 033018 (6 June 2023). https://doi.org/10.1117/1.JEI.32.3.033018
Received: 23 November 2022; Accepted: 19 May 2023; Published: 6 June 2023
Advertisement
Advertisement
RIGHTS & PERMISSIONS
Get copyright permission  Get copyright permission on Copyright Marketplace
KEYWORDS
Image quality

Image fusion

Education and training

Image restoration

Image processing

Visualization

Ablation

Back to Top