基于移动麦克风阵列的室内声源定位方法研究

      近年来，与人工智能相关的技术发展迅速，机器人试听学也引起越来 越多的关注。声音作为机器人感知环境中的一个重要因素不容忽略，配备 有声音感知的机器人可以进行声源定位与追踪、自主探索环境、协助作业、 人机互动等。因此声源追踪技术在机器人中也备受关注。

      标定麦克风阵列参数可以帮助麦克风阵列在实际应用中更准确地捕捉 声音信号。为了填补多麦克风阵列参数可观测性全面分析的空缺，本文将 Graph SLAM（Simultaneous Localization and Mapping）框架与多麦克风阵列 参数和声源定位相关联，通过 Fisher 信息矩阵的方法分析未知参数（麦克风阵列的几何位置，欧拉角，阵列间的异步参数和声源的位置）的可观测 性，由此发现了未知参数可观测的条件与声源的运动轨迹和麦克风阵列的 配置密切相关，给多麦克风阵列参数标定和声源定位提供了理论基础和指导方向。

      本文针对不同结构的麦克风阵列和声源定位算法进行研究，在现有基于时延估计的定位算法基础上结合阵列结构提出几何估计法来定位声源。 可控波束形成算法因其搜索空间复杂导致算法计算量大，本文通过球形四 层网格搜索法减小计算量，利用机器人移动的特点补充声源定位系统中缺 乏的距离信息，使其可以准确定位室内声源坐标，并借助机器人操作系统 完成室内声源寻找功能，实验验证了方法的可靠性和精准度。

       In recent years, the rapid development of technologies related to artificial intelligence has garnered increasing attention towards auditory robotics. Sound, as a critical factor in robot environmental perception, cannot be overlooked. Robots equipped with sound perception capabilities can perform tasks such as sound source localization and tracking, autonomous environmental exploration, assistance in operations, and human-robot interaction. Thus, sound source tracking technology has received considerable focus in robotics.

        Calibrating microphone array parameters can aid in capturing sound signals more accurately in practical applications. To address the gap in comprehensive observability analysis of multi-microphone array parameters, this paper links the Graph SLAM (Simultaneous Localization and Mapping) framework with multimicrophone array parameters and sound source localization. It analyzes the observability of unknown parameters (the geometric location of the microphone array, Euler angles, asynchronous parameters between arrays, and the position of the sound source) through the Fisher information matrix method. This analysis reveals that the observability conditions of unknown parameters are closely related to the motion trajectory of the sound source and the configuration of the microphone array, providing a theoretical foundation and guidance for the calibration of multi-microphone array parameters and sound source localization.

        This paper investigates different structures of microphone arrays and sound source localization algorithms. Based on existing time-delay estimation localization algorithms, it proposes a geometric estimation method for sound source localization that incorporates array structure. The controllable beamforming algorithm, due to its complex search space leading to high computational demands, is refined through a spherical four-layer grid search method to reduce computational load. Utilizing the mobility of robots to supplement the distance information lacking in sound source localization systems enables precise indoor sound source positioning. With the aid of the robot operating system, indoor sound source finding functionality is achieved.Experiments validate the reliability and accuracy of the method.

[1] 刘京运. “机器人+”应用行动加快推动高质量发展[J]. 机器人产业, 2023(02): 42-47.
[2] 王昊. 基于声达时间差的移动机器人声源定位方法研究[D]. 青岛科技大学, 2021.
[3] FLYNN A M, BROOKS R A, WELLS III W M, et al. Squirt: The prototypical mobile robot for autonomous graduate students[J]. 1989.
[4] BROOKS R A, BREAZEAL C, MARJANOVIĆ M, et al. The Cog project: Building a humanoid robot[C]//International workshop on computation for metaphors, analogy, and agents. Berlin, Heidelberg: Springer Berlin Heidelberg, 1998: 52-87.
[5] BROOKS R A, STEIN L A. Building brains for bodies[J]. Autonomous Robots, 1994, 1: 7-25.
[6] IRIE R E. Robust sound localization: An application of an auditory perception system for a humanoid robot[D]. Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 1995.
[7] TAKANISHI A, MASUKAWA S, MORI Y, et al. Study on anthropomorphic auditory robot continuous localization of a sound source in horizontal plane[C]//11th Annual Conference of the Robotics Society of Japan, 1993: 793-796.
[8] TAKANISHI A, MASUKAWA S, MORI Y, et al. Development of an anthropomorphic auditory robot that localizes a sound direction[J]. Bulletin of the Centre for Informatics, 1995, 20: 24-32.
[9] NAGASHIMA K, YOSHIIKE T, KONNO A, et al. Attention-based interaction between human and the robot chiye[C]//Proceedings 6th IEEE International Workshop on Robot and Human Communication. RO-MAN'97 SENDAI. IEEE, 1997: 100-105.
[10] HUANG J, OHNISHI N, SUGIE N. Building ears for robots: sound localization and separation[J]. Artificial Life and Robotics, 1997, 1: 157-163.
[11] HUANG J, OHNISHI N, SUGIE N. Sound localization in reverberant environment based on the model of the precedence effect[J]. IEEE Transactions on Instrumentation and Measurement, 1997, 46(4): 842-846.
[12] HUANG J, SUPAONGPRAPA T, TERAKURA I, et al. A model-based sound localization system and its application to robot navigation[J]. Robotics and Autonomous Systems, 1999, 27(4): 199-209.
[13] ASONO F, ASOH H, MATSUI T. Sound source localization and signal separation for office robot "Jijo-2"[C]//Proceedings. 1999 IEEE/SICE/RSJ. International Conference on Multisensor Fusion and Integration for Intelligent Systems. MFI'99 (Cat. No. 99TH8480). IEEE, 1999: 243-248.
[14] MATSUI T, ASOH H, FRY J, et al. Integrated natural spoken dialogue system of Jijo-2 mobile robot for office services[C]//1999 AAAI.
[15] HASHIMOTO S, NARITA S, KASAHARA H, et al. Humanoid robot-development of an information assistant robot Hadaly[C]//Proceedings 6th IEEE International Workshop on Robot and Human Communication RO-MAN'97 SENDAI. IEEE, 1997: 106-111.
[16] OKUNO H G, NAKADAI K. Active audition for humanoid robots that can listen to three simultaneous talkers[J]. The Journal of the Acoustical Society of America, 2003, 113(4_Supplement): 2230-2230.
[17] KIM U-H, NAKADAI K, OKUNO H G. Improved sound source localization and front-back disambiguation for humanoid robots with two ears [C]//Recent Trends in Applied Artificial Intelligence: 26th International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems, IEA/AIE 2013, Amsterdam, The Netherlands, June 17-21, 2013. Proceedings 26. Springer Berlin Heidelberg, 2013: 282-291.
[18] PORTELLO A, DANES P, ARGENTIERI S. Active binaural localization of intermittent moving sources in the presence of false measurements[C]//2012 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2012: 3294-3299.
[19] KOSSYK I, NEUMANN M, MARTON Z-C. Binaural bearing only tracking of stationary sound sources in reverberant environment[C]//2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids), IEEE, 2015: 53-56.
[20] CHEN J C, YIP L, ELSON J, et al. Coherent acoustic array processing and localization on wireless sensor networks[J]. Proceedings of the IEEE, 2003, 91(8): 1154-1162.
[21] FAZENDA B, ATMOKO H, GU F, et al. Acoustic based safety emergency vehicle detection for intelligent transport systems[C]//2009 ICCAS-SICE. IEEE, 2009: 4250-4255.
[22] HOSHIBA K, WASHIZAKI K, WAKABAYASHI M, et al. Design of UAV-embedded microphone array system for sound source localization in outdoor environments[J]. Sensors, 2017, 17(11): 2535.
[23] MICHAUD S, FAUCHER S, GRONDIN F, et al. 3D localization of a sound source using mobile microphone arrays referenced by SLAM[C]//2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2020: 10402-10407.
[24] FAN X, LEE D, CHEN Y, et al. Acoustic collision detection and localization for robot manipulators[C]//2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2020: 9529-9536.
[25] BAI Y, GARG N, ROY N. Spidr: Ultra-low-power acoustic spatial sensing for micro-robot navigation[C]//Proceedings of the 20th Annual International Conference on Mobile Systems, Applications and Services. 2022: 99-113.
[26] MAJID A Y, VAN DER HORST C, VAN RIETBERGEN T, et al. Lightweight audio source localization for swarm robots[C]//2021 IEEE 18th Annual Consumer Communications & Networking Conference (CCNC). IEEE, 2021: 1-6.
[27] ZHANG T, ZHANG H, LI X, et al. AcousticFusion: Fusing sound source localization to visual SLAM in dynamic environments[C]//2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2021: 6868-6875.
[28] SAQIB U, JENSEN J R. A model-based approach to acoustic reflector localization with a robotic platform[C]//2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2020: 4499-4504.
[29] RYPKEMA N R, FISCHELL E M, SCHMIDT H. Memory-efficient approximate three-dimensional beamforming[J]. The Journal of the Acoustical Society of America, 2020, 148(6): 3467-3480.
[30] WADA K, SHIBATA T, SAITO T, et al. Psychological and social effects of one year robot assisted activity on elderly people at a health service facility for the aged[C]// Proceedings of the 2005 IEEE international conference on robotics and automation. IEEE, 2005: 2785-2790.
[31] MEZA I, RASCON C, FUENTES G, et al. On indexicality, direction of arrival of sound sources, and human-robot interaction[J]. Journal of robotics, 2016.
[32] DO H M, SHENG W, LIU M. An open platform of auditory perception for home service robots[C]//2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). 2015: 6161-6166.
[33] REVELEAU A, FERLAND F, LABBé M, et al. Visual representation of interaction force and sound source in a teleoperation user interface for a mobile robot[J]. Journal of Human-Robot Interaction, 2015, 4(2): 1-23.
[34] NISHIMUTA I, ITOYAMA K, YOSHII K, et al. Toward a quizmaster robot for speech-based multiparty interaction[J]. Advanced Robotics, 2015, 29(18): 1205-1219.
[35] CHEN H, LEU M C, YIN Z. Real-time multi-modal human–robot collaboration using gestures and speech[J]. Journal of Manufacturing Science and Engineering, 2022, 144(10): 101007.
[36] MEYER T S, JUNIOR P T A. Sound source localization and tracking for the@ home service robot[C]//II BRAHUR and III Brazilian Workshop on Service Robotics. 2019: 59-64.
[37] CHAKRABARTY S, HABETS E A. Broadband DOA estimation using convolutional neural networks trained with noise signals[C]//2017 IEEE Workshop on Applications of Signal Processing to Audio and Acoustics (WASPAA). IEEE, 2017: 136-140.
[38] DIAZ-GUERRA D, MIGUEL A, BELTRAN J R. Robust sound source tracking using SRP-PHAT and 3D convolutional neural networks[J]. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2020, 29: 300-311.
[39] CHEN G, XU Y. A sound source localization device based on rectangular pyramid structure for mobile robot[J]. Journal of Sensors, 2019.
[40] QIU Y, LI B, HUANG J, et al. An analytical method for 3-D sound source localization based on a five-element microphone array[J]. IEEE Transactions on Instrumentation and Measurement, 2022, 71: 1-14.
[41] NAKADAI K, TAKAHASHI T, OKUNO H G, et al. Design and implementation of robot audition system 'HARK'—Open source software for listening to three simultaneous speakers[J]. Advanced Robotics, 2010, 24(5-6): 739-761.
[42] GRONDIN F, LéTOURNEAU D, FERLAND F, et al. The ManyEars open framework: Microphone array open software and open hardware system for robotic applications[J]. Autonomous Robots, 2013, 34: 217-232.
[43] GRONDIN F, MICHAUD F. Lightweight and optimized sound source localization and tracking methods for open and closed microphone array configurations[J]. Robotics and Autonomous Systems, 2019, 113: 63-80.
[44] GRONDIN F, LéTOURNEAU D, GODIN C, et al. ODAS: Open embedded audition system[J]. Frontiers in Robotics and AI, 2022, 9: 854444.
[45] PERRODIN F, NIKOLIC J, BUSSET J, et al. Design and calibration of large microphone arrays for robotic applications[C]//2012 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2012: 4596-4601.
[46] MIURA H, YOSHIDA T, NAKAMURA K, et al. SLAM-based online calibration of asynchronous microphone array for robot audition[C]//2011 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2011: 524-529.
[47] MIURA H, YOSHIDA T, NAKAMURA K, et al. SLAM-based online calibration for asynchronous microphone array[J]. Advanced Robotics, 2012, 26(17): 1941-1965.
[48] SU D, KONG H, SUKKARIEH S, et al. Necessary and sufficient conditions for observability of SLAM-based TDOA sensor array calibration and source localization[J]. IEEE Transactions on Robotics, 2021, 37(5): 1451-1468.
[49] SU D, VIDAL-CALLEJA T, MIRO J V. Simultaneous asynchronous microphone array calibration and sound source localization[C]//2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2015: 5561-5567.
[50] SU D, VIDAL-CALLEJA T, MIRO J V. Asynchronous microphone arrays calibration and sound source tracking[J]. Autonomous Robots, 2020, 44(2): 183-204.
[51] DAN K, ITOYAMA K, NISHIDA K, et al. Calibration of a microphone array based on a probabilistic model of microphone positions [C]//International Conference on Industrial, Engineering and Other Applications of Applied Intelligent Systems. Cham: Springer International Publishing, 2020: 614-625.
[52] PLINGE A, FINK G A, GANNOT S. Passive online geometry calibration of acoustic sensor networks[J]. IEEE Signal Processing Letters, 2017, 24(3): 324-328.
[53] WOŹNIAK S, KOWALCZYK K. Passive joint localization and synchronization of distributed microphone arrays[J]. IEEE Signal Processing Letters, 2018, 26(2): 292-296.
[54] GALA D, LINDSAY N, SUN L. Realtime active sound source localization for unmanned ground robots using a self-rotational bi-microphone array[J]. Journal of Intelligent & Robotic Systems, 2019, 95: 935-954.
[55] SUGIYAMA C, ITOYAMA K, NISHIDA K, et al. Simultaneous calibration of positions, orientations, and time offsets, among multiple microphone arrays[C]//2021 IEEE International Conference on Autonomous Systems (ICAS). IEEE, 2021: 1-5.
[56] 李威燃. 基于移动机器人平台的声源定位技术研究[D]. 浙江大学, 2019.
[57] 徐志夺. 基于小波变换的语音增强算法研究[D]. 哈尔滨工程大学, 2014.
[58] HUANG S, DISSANAYAKE G. A critique of current developments in simultaneous localization and mapping[J]. International Journal of Advanced Robotic Systems, 2016, 13(5): 1729881416669482.
[59] WANG Z, DISSANAYAKE G. Observability analysis of SLAM using Fisher information matrix [C]//2008 10th International Conference on Control, Automation, Robotics and Vision. IEEE, 2008: 1242-1247.
[60] SICILIANA B, SCIAVICOO L, VILLANI L, et al. Robotics modeling, planning and control[J]. Springer-Verlag London Limited, 2009.
[61] 冯仕轩. 基于麦克风阵列的无人机定位方法研究[D]. 西安电子科技大学, 2019.
[62] 韩智轩. 基于波束形成的机器人声源定位方法研究[D]. 南京信息工程大学, 2021.
[63] 刘永存, 贺慧勇. 声音时延测量的数字电路实现与应用[J]. 仪表技术与传感器, 2020.
[64] 袁列萍. 基于声信号处理的被动目标探测定位方法研究[D]. 西安电子科技大学, 2019.
[65] 王铭喆, 邹捷源, 张国才等. 基于GCC算法的声源识别与三维空间实时定位[J]. 物联网技术, 2023, 13(10): 42-44+48.
[66] 杨思琪, 田杨萌, 王彩霞等. TDOA声源定位中阵列阵型对误差的影响[J]. 太赫兹科学与电子信息学报, 2023, 21(01): 50-57.
[67] 段丽萍. 基于 TDOA 算法的四麦克风阵列三维声源定位研究[J]. 2014
[68] 陈颖睿. 麦克风阵列波束成形算法研究与实现[D]. 南京: 南京邮电大学, 2020.
[69] 黄晓琴. 基于移动机器人的声源定位与跟踪方法研究[D]. 武汉理工大学, 2020.
[70] 张二亮, 曹雪林, 朱松. 快速 SRP-PHAT多声源定位算法[J]. 重庆理工大学学报 (自然科学), 2023, 37(9): 167-172.
[71] 刘海涛, 陈永华, 林艳明等. 基于TDOA多声源定位的虚假声源消除方法[J]. 振动.测试与诊断, 2021, 41(02): 319-326+413-414.
[72] 王小怀, 伊树彬, 蔡洛思. 基于TDOA技术及几何模型的声源精准定位[J]. 中国新技术新产品, 2022(22): 29-31+84.
[73] 吉爱国,刘伟平,刘志强等.麦克风阵列近场声源定位与跟踪系统[J]. 信息记录材料, 2019, 20(07): 149-152.
[74] 刘生. 基于 SRP-PHAT 的实时声源定位算法设计与实现[J]. 软件导刊, 2021.
[75] 黄静, 胡馨月. 基于麦克风阵列的室内三维声源定位优化算法[J]. 计算机系统应用, 2021, 30(09): 212-218.
[76] 杜鹏. 基于六轴传感器MPU6050的一种低功耗姿态及深度测量装置[J]. 舰船电子工程, 2022, 42(6): 168-170.
[77] RAFIQ A A, ROHMAN W N, RIYANTO S D. Development of a simple and low-cost smartphone gimbal with MPU-6050 sensor[J]. Journal of Robotics and Control (JRC), 2020, 1(4): 136-140.
[78] SULTAN J M, ZANI N H, AZUANI M, et al. Analysis of inertial measurement accuracy using complementary filter for MPU6050 sensor[J]. Jurnal Kejuruteraan, 2022, 34(5): 959-964.