基于脑电反应的空间听觉敏感性研究

空间听觉是人类声音定位的心理学基础，它在人类日常生活以及工程技术中扮演重要的角色。近年来，围绕听觉的空间方位特征所进行的研究是人们关注的焦点。听觉器官主要利用双耳的听觉线索来确定声源的空间方位，两侧大脑皮层的听觉中枢通过对这些线索进行综合处理，形成对声源的空间感知。这种空间感知能力可以帮助人们在听音过程中获取足够全面的声场内容，从而能够对周围的环境做出正确的理解和判断。此外，通过两侧听觉皮层的神经回路来区分空间上不同的声源信号，也就是测量人耳对于不同空间方位声源的辨别能力，对于感知周围环境信息、再现三维声场等具有重要意义。

以往的研究多采用行为学方法测量人耳对空间声源位置的辨别能力，而声源位置信息在听觉皮层中的编码方式以及敏感性表征形式尚不清楚。为了探究听觉空间信息的皮层表征机制，本研究使用失匹配负波成分（Mismatch Negativity，MMN）作为空间听觉敏感性的检测指标，使用Oddball范式测量不同空间方位声源刺激所引发的脑波变化。实验设置两种参考方位的刺激条件（分别对应前、右侧声学空间），每个参考方位下设置三种不同的偏差角度，以此探究参考方位和偏差角度对于声源辨别能力的影响。此外，在每个参考方位下对最小可听角的行为数据进行测量，以量化不同位置声源的空间分辨能力。两种测量方式揭示了前注意MMN反应与行为学阈值之间的联系。实验结果显示：（1）参考方位和偏差角度的变化都会诱发出MMN，且MMN振幅随着偏差角度的增大而增强，潜伏期随着声源参考方位右偏而延长；（2）ERP和行为学的实验结果一致显示人耳对于空间不同方位声源的辨别能力可达5°分辨率。本文的研究结果有助于进一步理解空间听觉定位的前意识处理机制，为虚拟现实环境下三维声场再现以及听觉脑机接口的发展提供了一定的神经生理学依据。

[1]ZIMMERMANN J F, MOSCOVITCH M, ALAIN C. Long-term memory biases auditory spatial attention[J]. Journal of Experimental Psychology: Learning, Memory, and Cognition, 2017, 43(10): 1602.
[2]ZHOU Z, CHEOK A D, YANG X, et al. An experimental study on the role of 3D sound in augmented reality environment[J]. Interacting with Computers, 2004, 16(6): 1043-1068.
[3]PULKKI V, HIRVONEN T. Localization of virtual sources in multichannel audio reproduction[J]. Speech and Audio Processing, IEEE Transactions on, 2005, 13(1): 105-119.
[4]LEWKOWICZ R, STRÓŻAK P, BAŁAJ B, et al. Auditory verbal working memory load effects on a simulator-induced spatial disorientation event[J]. Aerospace Medicine and Human Performance, 2019, 90(6): 531-539.
[5]YANG J, SASIKUMAR P, BAI H, et al. The effects of spatial auditory and visual cues on mixed reality remote collaboration[J]. Journal on Multimodal User Interfaces, 2020, 14(4): 337-352.
[6]OKUNO H G, NAKADAI K, LOURENS T, et al. Sound and visual tracking for humanoid robot[J]. Applied Intelligence, 2004, 20(3): 253-266.
[7]CHOI Y C, KIM Y H. Near field impulsive source localization in a noisy environment[J]. Journal of Sound and Vibration, 2007, 303(1): 209-220.
[8]VAN DE BOGAERT T, WOUTERS J, KLASEN T J, et al. Distortion of interaural time cues by directional noise reduction systems in modern digital hearing aids[C]//Applications of Signal Processing to Audio and Acoustics, 2005. IEEE Workshop on. IEEE, 2005: 57-60.
[9]HU H, EWERT S D, MCALPINE D, et al. Differences in the temporal course of interaural time difference sensitivity between acoustic and electric hearing in amplitude modulated stimuli[J]. The Journal of the Acoustical Society of America, 2017, 141(3): 1862-1873.
[10]PANNIELLO M, KING A J, DAHMEN J C, et al. Local and global spatial organization of interaural level difference and frequency preferences in auditory cortex[J]. Cerebral Cortex, 2018, 28(1): 350-369.
[11]MILLS A W. On the minimum audible angle[J]. The Journal of the Acoustical Society of America, 1958, 30(4): 237-246.
[12]YU G, WU R, LIU Y, et al. Near-field head-related transfer-function measurement and database of human subjects[J]. The Journal of the Acoustical Society of America, 2018, 143(3): EL194-EL198.
[13]BRINKMANN F, LINDAU A, WEINZIERL S, et al. A high resolution and full-spherical head-related transfer function database for different head-above-torso orientations[J]. Journal of the Audio Engineering Society, 2017, 65(10): 841-848.
[14]RICHTER J G, FELS J. On the influence of continuous subject rotation during high-resolution head-related transfer function measurements[J]. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2019, 27(4): 730-741.
[15]SINGH R P, JAVAID M, KATARIA R, et al. Significant applications of virtual reality for COVID-19 pandemic[J]. Diabetes & Metabolic Syndrome: Clinical Research & Reviews, 2020, 14(4): 661-664.
[16]FARSHID M, PASCHEN J, ERIKSSON T, et al. Go boldly!: Explore augmented reality (AR), virtual reality (VR), and mixed reality (MR) for business[J]. Business Horizons, 2018, 61(5): 657-663.
[17]WIKMAN P, SAHARI E, SALMELA V, et al. Breaking down the cocktail party: Attentional modulation of cerebral audiovisual speech processing[J]. Neuroimage, 2021, 224: 117365.
[18]BLAUERT J. Spatial hearing: the psychophysics of human sound localization[M]. MIT Press, 1997.
[19]RASPAUD M, VISTE H. Binaural source localization by joint estimation of ILD and ITD[J]. IEEE Transactions on Audio Speech & Language Processing, 2009, 18(1): 68-77.
[20]JIANG J, XIE B, MAI H, et al. The role of dynamic cue in auditory vertical localisation[J]. Applied Acoustics, 2019, 146: 398-408.
[21]NÄÄTÄNEN R, PAAVILAINEN P, RINNE T, et al. The mismatch negativity (MMN) in basic research of central auditory processing: a review[J]. Clinical Neurophysiology, 2007, 118(12): 2544-2590.
[22]ZHANG L I, BAO S, MERZENICH M M. Disruption of primary auditory cortex by synchronous auditory inputs during a critical period[J]. Proceedings of the National Academy of Sciences, 2002, 99(4): 2309-2314.
[23]LEE Y, PARK Y, PARK Y. Newly designed HRTF measuring system[C]//2009 ICCAS-SICE. IEEE, 2009: 1781-1784.
[24]WEN ZHANG, MENGQIU ZHANG, RODNEY A. KENNEDY, el al. On High-Resolution Head-Related Transfer Function Measurements: An Efficient Sampling Scheme[J]. Audio, Speech, and Language Processing. 2012, 20(2): 575-584.
[25]HONDA A, SHIBATA H, GYOBA J, et al. Transfer effects on sound localization performances from playing a virtual three - dimensional auditory game [J]. Applied Acoustics, 2007, 68(8): 885- 896.
[26]BILL G. Hrtf measurements of a KEMAR dummy-head microphone[J]. MIT Media Lab. Perceptual Computing-Technical Report, 1994, 280: 1-7.
[27]ALGAZI V R, DUDA R O, THOMPSON D M, et al. The cipic hrtf database[C]// Proceedings of the 2001 IEEE Workshop on the Applications of Signal Processing to Audio and Acoustics (Cat. No. 01TH8575). IEEE, 2001: 99-102.
[28]http://www.ircam.fr/equipes/salles/listen/index.html
[29]CHRISTENSEN F, JENSEN C B, MØLLER H. The design of VALDEMAR-an artificial head for binaural recording purposes[C]//Audio Engineering Society Convention 109. Audio Engineering Society, 2000.
[30]DEOUELL L Y, BENTIN S, SOROKER N. Electrophysiological evidence for an early (pre-attentive) information processing deficit in patients with right hemisphere damage and unilateral neglect[J]. Brain, 2000, 123(2): 353-365.
[31]DEOUELL L Y, HELLER A S, MALACH R, et al. Cerebral responses to change in spatial location of unattended sounds[J]. Neuron, 2007, 55(6): 985-996.
[32]GILKEY R H, GOOD M D, ERICSON M A, et al. A pointing technique for rapidly collecting localization responses in auditory research[J]. Behavior Research Methods, Instruments, & Computers, 1995, 27(1): 1-11.
[33]FREIGANG C, SCHMIEDCHEN K, NITSCHE I, et al. Free-field study on auditory localization and discrimination performance in older adults[J]. Experimental Brain Research, 2014, 232(4): 1157-1172.
[34]AGGIUS-VELLA E, KOLARIK A J, GORI M, et al. Comparison of auditory spatial bisection and minimum audible angle in front, lateral, and back space[J]. Scientific Reports, 2020, 10(1): 1-9.
[35]温晓慧,王宁宇,张娟.水平方位声源位置改变诱发健康青年人失匹配负波研究[J]. 听力学及言语疾病杂志,2015,23(4):338-342.
[36]谢菠荪,钟小丽,饶丹,等.头相关传输函数数据库及其特性分析[J].中国科学: G 辑,2006,36(5):464-479.
[37]王超, 胡剑凌. 基于 HRTF 的虚拟三维空间环绕声耳机重放[J]. 信息技术, 2009 (1): 39-41.
[38]吴杰,田霄海.基于中国人工头模的 HRTF 数据库设计与实现[J].計算機與現代化, 2011, 2011(12): 92-96.
[39]FENG WU, GAO XIA, LI ZHI-HONG, et al. Devising and initial realization of testing hearing system for sound location[J]. J Fourth Military Med Uni, 2001, 22(7):656-658.
[40]俞胜锋. 基于脑电的双耳听觉定位的初步研究[D]. 广州:华南理工大学,2019.
[41]HAN Y, CHEN F. Minimum audible movement angle in virtual auditory environment: effect of stimulus frequency[C]//2019 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR). IEEE, 2019: 175-178.
[42]ZHANG C Y, XIE B S. Platform for dynamic virtual auditory environment real-time rendering system[J]. Chinese Science Bulletin, 2013, 58(3): 316–327.
[43]SALADIN K S. Human anatomy[M]. Rex Bookstore, Inc., 2005.
[44]VON BARTHELD C S, BAHNEY J, HERCULANO‐HOUZEL S. The search for true numbers of neurons and glial cells in the human brain: A review of 150 years of cell counting[J]. Journal of Comparative Neurology, 2016, 524(18): 3865-3895.
[45]HENRY J C. Electroencephalography: basic principles, clinical applications, and related fields[J]. Neurology, 2006, 67(11): 2092-2092-a.
[46]GRAY H. Anatomy of the human body[M]. Lea & Febiger, 1878.
[47]HAAS L F. Hans berger (1873–1941), richard caton (1842–1926), and electroencephalography[J]. Journal of Neurology, Neurosurgery & Psychiatry, 2003, 74(1): 9-9.
[48]SAKHAVI S. Application of deep learning methods in brain-computer interface systems[D]. National University of Singapore (Singapore), 2017.
[49]WALTER W, COOPER R, ALDRIDGE V J, et al. Contingent negative variation: an electric sign of sensori-motor association and expectancy in the human brain[J]. nature, 1964, 203(4943): 380-384.
[50]Event-related potentials: A methods handbook[M]. MIT press, 2005.
[51]HINOJOSA J A, Martı́n-Loeches M, Rubia F J. Event-related potentials and semantics: An overview and an integrative proposal[J]. Brain and Language, 2001, 78(1): 128-139.
[52]LUCK S J. An introduction to the event-related potential technique[M]. MIT Press, 2014.
[53]MUSIEK F E, CHERMAK G D, WEIHING J. Auditory training[M]. Plural Publishing Inc., 2014.
[54]SANJU H K, KUMAR P. Enhanced auditory evoked potentials in musicians: A review of recent findings[J]. Journal of Otology, 2016, 11(2): 63-72.
[55]SUTTON S, BRAREN M, ZUBIN J, et al. Evoked-potential correlates of stimulus uncertainty[J]. Science, 1965, 150(3700): 1187-1188.
[56]YAGI Y, COBURN K L, ESTES K M, et al. Effects of aerobic exercise and gender on visual and auditory P300, reaction time, and accuracy[J]. European journal of applied physiology and occupational physiology, 1999, 80(5): 402-408.
[57]POLICH J. Updating P300: an integrative theory of P3a and P3b[J]. Clinical neurophysiology, 2007, 118(10): 2128-2148.
[58]Detection of change: event-related potential and fMRI findings[M]. Norwell, MA:: Kluwer Academic Publishers, 2003.
[59]NIJBOER F, SELLERS E W, MELLINGER J, et al. A P300-based brain–computer interface for people with amyotrophic lateral sclerosis[J]. Clinical neurophysiology, 2008, 119(8): 1909-1916.
[60]FARWELL L A, DONCHIN E. Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials[J]. Electroencephalography and clinical Neurophysiology, 1988, 70(6): 510-523.
[61]HANSENNE M. Le potentiel évoqué cognitif P300 (II): variabilité interindividuelle et application clinique en psychopathologie[J]. Neurophysiologie Clinique/Clinical Neurophysiology, 2000, 30(4): 211-231.
[62]Li J, Pu J, Cui H, et al. An online P300 brain–computer interface based on tactile selective attention of somatosensory electrical stimulation[J]. Journal of Medical and Biological Engineering, 2019, 39(5): 732-738.
[63]FOLSTEIN J R, VAN PETTEN C. Influence of cognitive control and mismatch on the N2 component of the ERP: a review[J]. Psychophysiology, 2008, 45(1): 152-170.
[64]KUTAS M, HILLYARD S A. Reading senseless sentences: Brain potentials reflect semantic incongruity[J]. Science, 1980, 207(4427): 203-205.
[65]RUGG M D, CURRAN T. Event-related potentials and recognition memory[J]. Trends in Cognitive Sciences, 2007, 11(6): 251-257.
[66]NÄÄTÄNEN R, GAILLARD A W K, MÄNTYSALO S. Early selective-attention effect on evoked potential reinterpreted[J]. Acta Psychologica, 1978, 42(4): 313-329.
[67]NAGAI T, TADA M, KIRIHARA K, et al. Mismatch negativity as a “translatable” brain marker toward early intervention for psychosis: a review[J]. Frontiers in Psychiatry, 2013, 4: 115.
[68]NÄÄTÄNEN R, SHIGA T, ASANO S, et al. Mismatch negativity (MMN) deficiency: a break-through biomarker in predicting psychosis onset[J]. International Journal of Psychophysiology, 2015, 95(3): 338-344.
[69]VAITULEVICH S F, SHESTOPALOVA L B. Interhemisphere asymmetry of auditory evoked potentials in humans and mismatch negativity during sound source localization[J]. Neuroscience and Behavioral Physiology, 2010, 40(6): 629-638.
[70]PHILLIPS H N, BLENKMANN A, HUGHES L E, et al. Convergent evidence for hierarchical prediction networks from human electrocorticography and magnetoencephalography[J]. Cortex, 2016, 82: 192-205.
[71]TADA M, KIRIHARA K, MIZUTANI S, et al. Mismatch negativity (MMN) as a tool for translational investigations into early psychosis: a review[J]. International Journal of Psychophysiology, 2019, 145: 5-14.
[72]LUCK S J. An introduction to the event-related potential technique[M]. MIT Press, 2014.
[73]CALDENHOVE S, BORGHANS L, BLOKLAND A, et al. Role of acetylcholine and serotonin in novelty processing using an oddball paradigm[J]. Behavioural Brain Research, 2017, 331: 199-204.
[74]GILKEY R, ANDERSON T R. Binaural and spatial hearing in real and virtual environments[M]. Psychology Press, 2014.
[75]KOHLRAUSCH A, BRAASCH J, KOLOSSA D, et al. An introduction to binaural processing[C]//The technology of binaural listening. Springer, Berlin, Heidelberg, 2013: 1-32.
[76]BLAUERT J, BRAASCH J. Modeling binaural processing: What next?[C]// Proceedings of Meetings on Acoustics 164ASA. Acoustical Society of America, 2012,18(1): 015005.
[77]NODAL F R, BAJO V M, KING A J. Plasticity of spatial hearing: behavioural effects of cortical inactivation[J]. The Journal of physiology, 2012, 590(16): 3965-3986.
[78]谢菠荪,孟庆林.听觉障碍患者的空间听觉[J].应用声学,2018,5.
[79]PARISE C V, KNORRE K, ERNST M O. Natural auditory scene statistics shapes human spatial hearing[J]. Proceedings of the National Academy of Sciences, 2014, 111(16): 6104-6108.
[80]XIE B. Head-related transfer function and virtual auditory display[M]. J. Ross Publishing, 2013.
[81]罗万伯. 现代多媒体技术应用教程[J]. 2004.
[82]WALL J. Post-cochlear auditory modelling for sound localisation using bio-inspired techniques[D]. University of Ulster, 2010.
[83]FEDDERSEN W E, SANDEL T T, TEAS D C, et al. Localization of high‐frequency tones[J]. the Journal of the Acoustical Society of America, 1957, 29(9): 988-991.
[84]IIDA K. HRTF and sound localization in the horizontal plane[C]//Head-Related Transfer Function and Acoustic Virtual Reality. Springer, Singapore, 2019: 15-24.
[85]JIN C, CORDEROY A, CARLILE S, et al. Spectral cues in human sound localization[J]. Advances in Neural Information Processing Systems, 1999, 12.
[86]WIGHTMAN F L, KISTLER D J. Headphone simulation of free‐field listening. II: Psychophysical validation[J]. The Journal of the Acoustical Society of America, 1989, 85(2): 868-878.
[87]MØLLER H, HAMMERSHØI D, JENSEN C B, et al. Transfer characteristics of headphones measured on human ears[J]. Journal of the Audio Engineering Society, 1995, 43(4): 203-217.
[88]MORIMOTO M, ANDO Y. On the simulation of sound localization[J]. Journal of the Acoustical Society of Japan (e), 1980, 1(3): 167-174.
[89]PERROTT D R, SABERI K. Minimum audible angle thresholds for sources varying in both elevation and azimuth[J]. The Journal of the Acoustical Society of America, 1990, 87(4): 1728-1731.
[90]LITOVSKY R Y. Developmental changes in the precedence effect: estimates of minimum audible angle[J]. The Journal of the Acoustical Society of America, 1997, 102(3): 1739-1745.
[91]KOISTINEN S, RINNE T, CEDERSTRÖM S, et al. Effects of significance of auditory location changes on event related brain potentials and pitch discrimination performance[J]. Brain Research, 2012, 1427: 44-53.
[92]COLIN C, RADEAU M, SOQUET A, et al. Electrophysiology of spatial scene analysis: the mismatch negativity (MMN) is sensitive to the ventriloquism illusion[J]. Clinical Neurophysiology, 2002, 113(4): 507-518.
[93]RÖTTGER S, SCHRÖGER E, GRUBE M, et al. Mismatch negativity on the cone of confusion[J]. Neuroscience Letters, 2007, 414(2): 178-182.
[94]SHINN-CUNNINGHAM B G, SANTARELLI S, KOPCO N. Tori of confusion: Binaural localization cues for sources within reach of a listener[J]. The Journal of the Acoustical Society of America, 2000, 107(3): 1627-1636.
[95]RÖTTGER S, SCHRÖGER E, GRUBE M, et al. Mismatch negativity on the cone of confusion[J]. Neuroscience Letters, 2007, 414(2): 178-182.
[96]ZHAO G, ZHANG Y, GE Y. Frontal EEG asymmetry and middle line power difference in discrete emotions[J]. Frontiers in Behavioral Neuroscience, 2018: 225.
[97]AMENEDO E, ESCERA C. The accuracy of sound duration representation in the human brain determines the accuracy of behavioural perception[J]. European Journal of Neuroscience, 2000, 12(7): 2570-2574.
[98]NÄÄTÄNEN R, SCHRÖGER E, KARAKAS S, et al. Development of a memory trace for a complex sound in the human brain[J]. Neuroreport: An International Journal for the Rapid Communication of Research in Neuroscience, 1993.