基于声学和光学融合的飞行目标检测系统

本研究提出了一种基于声光融合的无人机目标检测和跟踪系统，旨在解决当前社会治安领域迫切需要高效无人机监控系统的问题。随着小型商用无人机的普及，传统的监控手段已经无法满足日益增长的无人机监控需求。本文首先对目前已有无人机检测手段进行调研，分析了现有工作的缺陷和无人机检测任务的难点，然后设计提出的一种结合了多个传感器模块的检测系统，包括麦克风阵列、相机和激光雷达。64 通道的麦克风阵列提供了 360 度的监视和高信噪比的声源估计，而远距离激光雷达和长焦相机能够在狭窄的视场内进行精确的目标定位。本系统采用声学和光学感知特性各自的优势，实现了从粗定位调整为精定位的整体定位策略，既能实现大范围的初步检测，又能进行高精度的定位跟踪。为了克服传统声源定位方法在面对环境噪声干扰时的缺陷。本研究还针对性的设计训练了一种用于环境降噪的深度学习模型，有效地选择了从麦克风采集的信号里过滤掉干扰噪声，并保留无人机目标的声音信号以提取有效的声学特征，减小声学估测误差。最后实地进行的实测试验验证了所提出的系统方法的有效性和优势，证明该无人机监控系统具有大 FOV、精度高、三维定位等优点。该系统还是首个多模态的三维跟踪系统，可以期待将为社会治安监控等领域提供有力的支持，具有广泛的应用前景。

[1] Commercial UAV Market Analysis By Product[EB/OL]. https://www.grandviewresearch.com/industry-analysis/commercial-uavmarket.

[2] Gatwick Drone Disruption Cost Airport Just £1.4 m[EB/OL]. https://www.theguardian.com/uk-news/2019/jun/18/gatwick-drone-disruption-cost-airport-just-14m.

[3] LIU W, ANGUELOV D, ERHAN D, et al. Ssd: Single shot multibox detector[C]//Europeanconference on computer vision. Springer, 2016: 21-37.

[4] REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: Unified, real-time objectdetection[C]//Proceedings of the IEEE conference on computer vision and pattern recognition.2016: 779-788.

[5] MOLCHANOV P, HARMANNY R I, DE WIT J J, et al. Classification of small UAVs and birdsby micro-Doppler signatures[J]. International Journal of Microwave and Wireless Technologies,2014, 6(3-4): 435-444.

[6] THOMAS A, LEBOUCHER V, COTINAT A, et al. UAV localization using panoramic thermalcameras[C]//International Conference on Computer Vision Systems. Springer, 2019: 754-767.

[7] ZUNINO A, CROCCO M, MARTELLI S, et al. Seeing the sound: A new multimodal imagingdevice for computer vision[C]//Proceedings of the IEEE International Conference on ComputerVision Workshops. 2015: 6-14.

[8] SEDUNOV A, SUTIN A, SEDUNOV N, et al. Passive acoustic system for tracking low-flyingaircraft[J]. IET Radar, Sonar & Navigation, 2016, 10(9): 1561-1568.

[9] JEON S, SHIN J W, LEE Y J, et al. Empirical study of drone sound detection in real-lifeenvironment with deep neural networks[C]//2017 25th European Signal Processing Conference(EUSIPCO). IEEE, 2017: 1858-1862.

[10] VALIN J M, MICHAUD F, ROUAT J, et al. Robust sound source localization using a microphone array on a mobile robot[C]//Proceedings 2003 IEEE/RSJ International Conference onIntelligent Robots and Systems (IROS 2003)(Cat. No. 03CH37453): volume 2. IEEE, 2003:1228-1233.

[11] Anti-Drone System Overview and Technology Comparison[EB/OL]. 2016

[2021-12-16]. https://anti-drone.eu/blog/anti-drone-publications/anti-drone-system-overview-and-technology-comparison.html.

[12] CHRISTNACHER F, HENGY S, LAURENZIS M, et al. Optical and acoustical UAV detection[C]//Electro-Optical Remote Sensing X: volume 9988. SPIE, 2016: 83-95.

[13] TONG J, XIE W, HU Y H, et al. Estimation of low-altitude moving target trajectory using singleacoustic array[J]. The Journal of the Acoustical Society of America, 2016, 139(4): 1848-1858.

[14] SCHMIDT R. Multiple emitter location and signal parameter estimation[J]. IEEE transactions on antennas and propagation, 1986, 34(3): 276-280.

[15] NA Y, GUO Y, FU Q, et al. Cross array and rank-1 MUSIC algorithm for acoustic highway lanedetection[J]. IEEE Transactions on Intelligent Transportation Systems, 2016, 17(9): 2502-2514.

[16] LIU H, WEI Z, CHEN Y, et al. Drone detection based on an audio-assisted camera array[C]//2017 IEEE Third International Conference on Multimedia Big Data (BigMM). IEEE, 2017:402-406.

[17] 刘芳; 浦昭辉; 张帅超. 基于注意力特征融合的无人机多目标跟踪算法[J]. 控制与决策,2021: 1-9.

[18] 叶涛; 赵宗扬; 柴兴华; 张俊. 基于改进卷积神经网络的非合作无人机检测应用[J]. 科学技术与工程, 2021, 21: 14245-14250.

[19] JANSSON A, HUMPHREY E, MONTECCHIO N, et al. Singing voice separation with deepu-net convolutional networks[Z]. 2017.

[20] GUVENC I, KOOHIFAR F, SINGH S, et al. Detection, tracking, and interdiction for amateurdrones[J]. IEEE Communications Magazine, 2018, 56(4): 75-81.

[21] FIORANELLI F, RITCHIE M, GRIFFITHS H, et al. Classification of loaded/unloaded microdrones using multistatic radar[J]. Electronics Letters, 2015, 51(22): 1813-1815.

[22] DROZDOWICZ J, WIELGO M, SAMCZYNSKI P, et al. 35 GHz FMCW drone detectionsystem[C]//2016 17th International Radar Symposium (IRS). IEEE, 2016: 1-4.

[23] CARIS M, JOHANNES W, STANKO S, et al. Millimeter wave radar for perimeter surveillanceand detection of MAVs (Micro Aerial Vehicles)[C]//2015 16th International Radar Symposium(IRS). IEEE, 2015: 284-287.

[24] KLARE J, BIALLAWONS O, CERUTTI-MAORI D. Detection of UAVs using the MIMOradar MIRA-CLE Ka[C]//Proceedings of EUSAR 2016: 11th European Conference on Synthetic Aperture Radar. VDE, 2016: 1-4.

[25] NGUYEN P, RAVINDRANATHA M, NGUYEN A, et al. Investigating cost-effective RF-baseddetection of drones[C]//Proceedings of the 2nd workshop on micro aerial vehicle networks,systems, and applications for civilian use. 2016: 17-22.

[26] HALUZA M, ČECHÁK J. Analysis and decoding of radio signals for remote control of drones[C]//2016 New Trends in Signal Processing (NTSP). IEEE, 2016: 1-5.

[27] DALAL N, TRIGGS B. Histograms of oriented gradients for human detection[C]//2005 IEEEcomputer society conference on computer vision and pattern recognition (CVPR’05): volume 1.Ieee, 2005: 886-893.

[28] NALAMATI M, KAPOOR A, SAQIB M, et al. Drone detection in long-range surveillancevideos[C]//2019 16th IEEE International Conference on Advanced Video and Signal BasedSurveillance (AVSS). IEEE, 2019: 1-6.

[29] UNLU E, ZENOU E, RIVIERE N, et al. Deep learning-based strategies for the detection andtracking of drones using several cameras[J]. IPSJ Transactions on Computer Vision and Applications, 2019, 11(1): 1-13.

[30] ZHENG Y, CHEN Z, LV D, et al. Air-to-air visual detection of micro-uavs: An experimentalevaluation of deep learning[J]. IEEE Robotics and automation letters, 2021, 6(2): 1020-1027.

[31] ZHOU Y, TUZEL O. Voxelnet: End-to-end learning for point cloud based 3d object detection[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2018:4490-4499.

[32] QI C R, SU H, MO K, et al. Pointnet: Deep learning on point sets for 3d classification and segmentation[C]//Proceedings of the IEEE conference on computer vision and pattern recognition.2017: 652-660.

[33] HAMMER M, HEBEL M, LAURENZIS M, et al. Lidar-based detection and tracking of smallUAVs[C]//Emerging Imaging and Sensing Technologies for Security and Defence III; and Unmanned Sensors, Systems, and Countermeasures: volume 10799. SPIE, 2018: 177-185.

[34] DOGRU S, MARQUES L. Drone detection using sparse lidar measurements[J]. IEEE Roboticsand Automation Letters, 2022, 7(2): 3062-3069.

[35] BENYAMIN M, GOLDMAN G H. Acoustic detection and tracking of a Class I UAS with asmall tetrahedral microphone array[R]. Army Research Lab Adelphi MD, 2014.

[36] CHANG X, YANG C, WU J, et al. A surveillance system for drone localization and trackingusing acoustic arrays[C]//2018 IEEE 10th Sensor Array and Multichannel Signal ProcessingWorkshop (SAM). IEEE, 2018: 573-577.

[37] [EB/OL]. 2022. https://www.droneshield.com/dronesentry.

[38] [EB/OL]. 2022. https://www.iai.co.il/p/eli-4030-drone-guard?gclid=.

[39] CARAPEZZA E M, LAYCOCK L, WHITE H J, et al. Unmanned/Unattended Sensors andSensor Networks XI; and Advanced Free-Space Optical Communication Techniques and Applications: volume 9647[M]. 2015.

[40] SHI X, YANG C, XIE W, et al. Anti-drone system with multiple surveillance technologies:Architecture, implementation, and challenges[J]. IEEE Communications Magazine, 2018, 56(4): 68-74.

[41] GENTILHO JR E, SCALASSARA P R, ABRÃO T. Direction-of-arrival estimation methods:A performance-complexity tradeoff perspective[J]. Journal of Signal Processing Systems, 2020,92(2): 239-256.

[42] CHIARIOTTI P, MARTARELLI M, CASTELLINI P. Acoustic beamforming for noise sourcelocalization–Reviews, methodology and applications[J]. Mechanical Systems and Signal Processing, 2019, 120: 422-448.

[43] SIJTSMA P. Acoustic beamforming for the ranking of aircraft noise[M]. National AerospaceLaboratory NLR, 2012.

[44] ow does Delay-and-Sum Beamforming in the Time Domain work[EB/OL]. https://www.gfaitech.com/knowledge/faq/delay-and-sum-beamforming-in-the-time-domain.

[45] RAYLEIGH L. XXXI. Investigations in optics, with special reference to the spectroscope[J].The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science, 1879, 8(49): 261-274.

[46] Wide Bandwidth, Low Noise, Precision Top Port SiSonic Microphone: SPH0642HT5H-1[M].Knowles Electronics, 2014.

[47] LIU Z, ZHANG F, HONG X. Low-cost retina-like robotic lidars based on incommensurablescanning[J]. IEEE/ASME Transactions on Mechatronics, 2021, 27(1): 58-68.

[48] VAN TREES H L. Optimum array processing: Part IV of detection, estimation, and modulationtheory[M]. John Wiley & Sons, 2002.

[49] OORD A V D, DIELEMAN S, ZEN H, et al. Wavenet: A generative model for raw audio[A].2016.

[50] WANG Q, MUCKENHIRN H, WILSON K, et al. Voicefilter: Targeted voice separation byspeaker-conditioned spectrogram masking[A]. 2018.

[51] SHEN J, PANG R, WEISS R J, et al. Natural tts synthesis by conditioning wavenet on melspectrogram predictions[C]//2018 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, 2018: 4779-4783.

[52] PASZKE A, GROSS S, CHINTALA S, et al. Automatic differentiation in pytorch[Z]. 2017.

[53] GRONDIN F, MICHAUD F. Lightweight and optimized sound source localization and trackingmethods for open and closed microphone array configurations[J]. Robotics and AutonomousSystems, 2019, 113: 63-80.