Face super resolution with a high frequency highway

Zeng, Dan1,2; Jiang, Wen1,2; Yan, Xiao1,2; Fu, Weibao1,2; Shen, Qiaomu1,2; Veldhuis, Raymond3; Tang, Bo1,2

2024-08-01

10.1049/ipr2.13195

IET IMAGE PROCESSING   影响因子和分区

1751-9659

1751-9667

["Face shape priors such as landmarks, heatmaps, and parsing maps are widely used to improve face super resolution (SR). It is observed that face priors provide locations of high-frequency details in key facial areas such as the eyes and mouth. However, existing methods fail to effectively exploit the high-frequency information by using the priors as either constraints or inputs. This paper proposes a novel high frequency highway (H2F${\\rm H}_2{\\rm F}$) framework to better utilize prior information for face SR, which dynamically decomposes the final SR face into a coarse SR face and a high frequency (HF) face. The coarse SR face is reconstructed from a low-resolution face via a texture branch, using only pixel-wise reconstruction loss. Meanwhile, the HF face is directly generated from face priors via an HF branch that employs the proposed inception-hourglass model. As a result, H2F${\\rm H}_2{\\rm F}$ allows the face priors to have a direct impact on the SR face by adding the outputs of both branches as the final result and provides an extra face editing function. Extensive experiments show that H2F${\\rm H}_2{\\rm F}$ significantly outperforms state-of-the-art face SR methods, is general for different texture branch models and face priors, and is robust to dataset mismatch and pose variations.","The method is the first to construct an HF face directly from the face priors via a high-frequency highway for face super-resolution, making it easy to understand the HF information gain. The method dynamically decomposes a final SR face into a coarse SR face and an HF face, making it possible to prevent the smoothing of HF details during learning. image"]

image enhancement image processing image restoration image resolution

SCI ; EI

英语

第一 ; 通讯

National Natural Science Foundation of China[62206123]

Computer Science ; Engineering ; Imaging Science & Photographic Technology

Computer Science, Artificial Intelligence ; Engineering, Electrical & Electronic ; Imaging Science & Photographic Technology

WOS:001285597400001

WILEY

Web of Science

1.Southern Univ Sci & Technol, Dept Comp Sci & Engn, Shenzhen, Peoples R China
2.Southern Univ Sci & Technol, Res Inst Trustworthy Autonomous Syst, Shenzhen, Peoples R China
3.Univ Twente, Fac EEMCS, Enschede, Netherlands

Zeng, Dan,Jiang, Wen,Yan, Xiao,et al. Face super resolution with a high frequency highway[J]. IET IMAGE PROCESSING,2024.

Zeng, Dan.,Jiang, Wen.,Yan, Xiao.,Fu, Weibao.,Shen, Qiaomu.,...&Tang, Bo.(2024).Face super resolution with a high frequency highway.IET IMAGE PROCESSING.

Zeng, Dan,et al."Face super resolution with a high frequency highway".IET IMAGE PROCESSING (2024).