Multimodal mixing convolutional neural network and transformer for Alzheimer's disease recognition

Chen，Junde1; Wang，Yun2; Zeb，Adnan3; Suzauddola，M. D.4; Wen，Yuxin1

2025

10.1016/j.eswa.2024.125321

Expert Systems with Applications   影响因子和分区

0957-4174

Early recognition of Alzheimer's disease (AD) and its precursor state, mild cognitive impairment (MCI), is pivotal in interrupting the progression of the disease and providing suitable treatment. Recent development in deep learning techniques has drawn great research attention for improving the efficacy of AD recognition. However, numerous current methods solely utilize data from a single auxiliary domain, limiting their ability to harness valuable intrinsic insights from multiple domains. To cope with the challenge, this paper is devoted to establishing an innovative multimodal medical data fusion model, termed as MMDF, to perform Alzheimer's disease recognition. Multimodal data including clinical records and medical images are used by the proposed approach, and backbone models are constructed using various data modalities. Specifically, a vision transformer model, which is termed as MRI_ViT, is tailored to recognize AD using brain magnetic resonance imaging (MRI) data. In parallel, a novel multi-scale attention-embedded one-dimensional (1D) convolutional neural network (MA-1DCNN) is devised for analyzing clinical records. Subsequently, these basic models are combined for creating a new data fusion model to recognize Alzheimer's disease. The experimental results reveal outstanding performance compared with state-of-the-art (SOTA) methods.

Alzheimer's disease recognition CNN Data fusion MRI Vision transformer

英语

其他

2-s2.0-85203535080

Scopus

1.Dale E. and Sarah Ann Fowler School of Engineering,Chapman University,Orange,92866,United States
2.School of Pharmacy,Chapman University,Irvine,92618,United States
3.Department of Computer Science and Engineering,Southern University of Science and Technology,Guangdong,Shenzhen,518000,China
4.School of Informatics,Xiamen University,Fujian,Xiamen,361005,China
5.

Chen，Junde,Wang，Yun,Zeb，Adnan,et al. Multimodal mixing convolutional neural network and transformer for Alzheimer's disease recognition[J]. Expert Systems with Applications,2025,259.

Chen，Junde,Wang，Yun,Zeb，Adnan,Suzauddola，M. D.,&Wen，Yuxin.(2025).Multimodal mixing convolutional neural network and transformer for Alzheimer's disease recognition.Expert Systems with Applications,259.

Chen，Junde,et al."Multimodal mixing convolutional neural network and transformer for Alzheimer's disease recognition".Expert Systems with Applications 259(2025).