Multi-Constraint Transferable Generative Adversarial Networks for Cross-Modal Brain Image Synthesis

Huang, Yawen1; Zheng, Hao1; Li, Yuexiang2; Zheng, Feng3; Zhen, Xiantong4; Qi, GuoJun5; Shao, Ling6; Zheng, Yefeng1

2024-05-01

10.1007/s11263-024-02109-4

INTERNATIONAL JOURNAL OF COMPUTER VISION   影响因子和分区

0920-5691

1573-1405

Recent progress in generative models has led to the drastic growth of research in image generation. Existing approaches show visually compelling results by learning multi-modal distributions, but they still lack realism, especially in certain scenarios like medical image synthesis. In this paper, we propose a novel Brain Generative Adversarial Network (BrainGAN) that explores GANs with multi-constraint and transferable property for cross-modal brain image synthesis. We formulate BrainGAN by introducing a unified framework with new constraints that can enhance modal matching, texture details and anatomical structure, simultaneously. We show how BrainGAN can learn meaningful tissue representations with rich variability of brain images. In addition to generating 3D volumes that are visually indistinguishable from real ones, we model adversarial discriminators and segmentors jointly, along with the proposed cost functions, which forces our networks to synthesize brain MRIs with realistic textures conditioned on anatomical structures. BrainGAN is evaluated on three public datasets, where it consistently outperforms the other state-of-the-art approaches by a large margin, advancing cross-modal synthesis of brain images both visually and practically.

Image synthesis Cross-modal Multi-constraint Generative adversarial network

SCI ; EI

英语

其他

Computer Science

Computer Science, Artificial Intelligence

WOS:001233690200003

SPRINGER

ENGINEERING

Web of Science

1.Jarvis Res Ctr, Tencent YouTu Lab, Shenzhen, Peoples R China
2.Guangxi Med Univ, Guangxi Key Lab Genom & Personalized Med, Med AI Res MARS Grp, Nanning 530021, Guangxi, Peoples R China
3.Southern Univ Sci & Technol, Shenzhen, Peoples R China
4.United Imaging Healthcare Co Ltd, Cent Res Inst, Beijing, Peoples R China
5.Univ Cent Florida, Orlando, FL USA
6.Univ Chinese Acad Sci, UCAS Terminus AI Lab, Beijing 100049, Peoples R China

Huang, Yawen,Zheng, Hao,Li, Yuexiang,et al. Multi-Constraint Transferable Generative Adversarial Networks for Cross-Modal Brain Image Synthesis[J]. INTERNATIONAL JOURNAL OF COMPUTER VISION,2024.

Huang, Yawen.,Zheng, Hao.,Li, Yuexiang.,Zheng, Feng.,Zhen, Xiantong.,...&Zheng, Yefeng.(2024).Multi-Constraint Transferable Generative Adversarial Networks for Cross-Modal Brain Image Synthesis.INTERNATIONAL JOURNAL OF COMPUTER VISION.

Huang, Yawen,et al."Multi-Constraint Transferable Generative Adversarial Networks for Cross-Modal Brain Image Synthesis".INTERNATIONAL JOURNAL OF COMPUTER VISION (2024).