Integration of cognitive tasks into artificial general intelligence test for large models

Qu，Youzhi1; Wei，Chen1; Du，Penghui1; Che，Wenxin1; Zhang，Chi1; Ouyang，Wanli2; Bian，Yatao3; Xu，Feiyang4; Hu，Bin5; Du，Kai6; Wu，Haiyan7; Liu，Jia8; Liu，Quanying1

2024-04-19

10.1016/j.isci.2024.109550

iScience   影响因子和分区

2589-0042

During the evolution of large models, performance evaluation is necessary for assessing their capabilities. However, current model evaluations mainly rely on specific tasks and datasets, lacking a united framework for assessing the multidimensional intelligence of large models. In this perspective, we advocate for a comprehensive framework of cognitive science-inspired artificial general intelligence (AGI) tests, including crystallized, fluid, social, and embodied intelligence. The AGI tests consist of well-designed cognitive tests adopted from human intelligence tests, and then naturally encapsulates into an immersive virtual community. We propose increasing the complexity of AGI testing tasks commensurate with advancements in large models and emphasizing the necessity for the interpretation of test results to avoid false negatives and false positives. We believe that cognitive science-inspired AGI tests will effectively guide the targeted improvement of large models in specific dimensions of intelligence and accelerate the integration of large models into human society.

Cognitive neuroscience Neuroscience Research methodology social sciences Social sciences

SCI

英语

第一 ; 通讯

2-s2.0-85189562652

Scopus

1.Department of Biomedical Engineering,Southern University of Science and Technology,Shenzhen,518055,China
2.Shanghai AI Laboratory,Shanghai,200232,China
3.Tencent AI lab,Shenzhen,518057,China
4.iFLYTEK AI Research,Hefei,230088,China
5.School of Medical Technology,Beijing Institute of Technology,Beijing,100081,China
6.Institute for Artificial Intelligence,Peking University,Beijing,100871,China
7.Centre for Cognitive and Brain Sciences and Department of Psychology,University of Macau,Macau,999078,China
8.Department of Psychology,Tsinghua University,Beijing,100084,China

Qu，Youzhi,Wei，Chen,Du，Penghui,et al. Integration of cognitive tasks into artificial general intelligence test for large models[J]. iScience,2024,27(4).

Qu，Youzhi.,Wei，Chen.,Du，Penghui.,Che，Wenxin.,Zhang，Chi.,...&Liu，Quanying.(2024).Integration of cognitive tasks into artificial general intelligence test for large models.iScience,27(4).

Qu，Youzhi,et al."Integration of cognitive tasks into artificial general intelligence test for large models".iScience 27.4(2024).