LDDMM-Face: Large deformation diffeomorphic metric learning for cross-annotation face alignment

Yang，Huilin1,2; Lyu，Junyan3; Cheng，Pujin4; Tam，Roger1; Tang，Xiaoying2

2024-10-01

10.1016/j.patcog.2024.110569

Pattern Recognition   影响因子和分区

0031-3203

We propose an innovative, flexible, and consistent cross-annotation face alignment framework, LDDMM-Face, the key contribution of which is a deformation layer that naturally embeds facial geometry in a diffeomorphic way. This enables and solves cross-annotation face alignment tasks that were impossible in the existing works. Instead of predicting facial landmarks via a heatmap or coordinate regression, we formulate the face alignment task in a diffeomorphic registration manner and predict momenta that uniquely parameterize the deformation between the initial boundary and true boundary. We then perform large deformation diffeomorphic metric mapping (LDDMM) simultaneously for curve and landmark to localize the facial landmarks. The novel embedding of LDDMM into a deep network allows LDDMM-Face to consistently annotate facial landmarks without ambiguity and flexibly handle various annotation schemes, and can even predict dense annotations from sparse ones. To the best of our knowledge, this is the first study to leverage learning-based diffeomorphic mapping for face alignment. Our method can be easily integrated into various face alignment networks. We extensively evaluate LDDMM-Face on five benchmark datasets: 300W, WFLW, HELEN, COFW-68, and AFLW. LDDMM-Face distinguishes itself with outstanding performance when dealing with within-dataset cross-annotation learning (sparse-to-dense) and cross-dataset learning (different training and testing datasets). In addition, LDDMM-Face shows promising results on the most challenging task of cross-dataset cross-annotation learning (different training and testing datasets with different annotations). Our codes are available at https://github.com/CRazorback/LDDMM-Face.

Deep learning Diffeomorphic mapping Face alignment Facial landmarks

EI

英语

通讯

20242116129071

Alignment ; Deep learning ; Forecasting ; Knowledge management ; Statistical tests

Surveying:405.3 ; Ergonomics and Human Factors Engineering:461.4 ; Mechanical Devices:601.1 ; Computer Applications:723.5 ; Information Retrieval and Use:903.3 ; Mathematical Statistics:922.2

ENGINEERING

2-s2.0-85193639656

Scopus

1.The University of British Columbia,Vancouver,251 - 2222 Health Sciences Mall,V6T 1Z3,Canada
2.Southern University of Science and Technology,Shenzhen,No. 1088 Xueyuan Road,518055,China
3.The University of Queensland,St Lucia,4067,Australia
4.The Univerisity of Hong Kong,Pok Fu Lam, Hong Kong SAR,Hong Kong

Yang，Huilin,Lyu，Junyan,Cheng，Pujin,et al. LDDMM-Face: Large deformation diffeomorphic metric learning for cross-annotation face alignment[J]. Pattern Recognition,2024,154.

Yang，Huilin,Lyu，Junyan,Cheng，Pujin,Tam，Roger,&Tang，Xiaoying.(2024).LDDMM-Face: Large deformation diffeomorphic metric learning for cross-annotation face alignment.Pattern Recognition,154.

Yang，Huilin,et al."LDDMM-Face: Large deformation diffeomorphic metric learning for cross-annotation face alignment".Pattern Recognition 154(2024).