基于音频的人体手势识别系统设计与实现

DESIGN AND IMPLEMENTATION OF ACOUSTIC BASED GESTURE RECOGNITION SYSTEM

马鸿

11849201

硕士

计算机技术领域工程

张进

2020-05-30

2020-07-20

哈尔滨工业大学

深圳

人体识别与人机交互在智能家居等商用场景有着迫切的需求和应用价值， 设计合理且成本低的人机交互方式是近年来的研究重点。当前已有的研究提出 的解决方案主要使用普通摄像头、深度摄像头以及可穿戴传感器。基于摄像头 解决方案的缺点在于无法应用在较为隐私的使用场景且容易被环境物体遮挡， 同时基于摄像头拍摄的视频内容的算法计算量大能耗较高；基于可穿戴传感器 的系统虽然计算量低且准确性较高，但要求用户必须穿戴特定的传感器设备， 增加了额外的硬件成本与使用成本。 本文提出的基于音频信号的人体手势识别系统，相比于现有的技术具有隐 私保护、被探测人体无需穿戴额外设备和低硬件成本等优点。本课题对基于音 频信号探测的基本原理进行了深入研究，讨论了不同的发射音频信号检测人体 时具有的不同特征，提出了基于信道估计的信号处理算法。同时针对在手势识 别应用场景中环境干扰以及用户个人动作特征对识别准确率的影响这两点问题， 提出了领域自适应的神经网络进行识别，相比于传统的神经网络结构，有效提 升了手势识别的准确率和系统的鲁棒性。 本论文的主要工作内容为：针对人体手势识别这一应用场景，分析了不同 的发射音频信号所能解调出的与手势相关的特征信息，理论分析了相应的信号 处理算法，设计了基于不同音频信号的手势识别系统；同时讨论了不同的发射 音频信号在人体手势识别时的理论精度，提出了基于信道估计的手势检测算法， 有效提升了手势检测的准确性。针对现阶段手势识别算法准确率存在被环境以 及用户个人动作特征干扰而降低的问题，提出了领域自适应神经网络网络用于 手势识别，有效提升了针对不同环境和不同用户进行识别时的识别准确率。本 工作完整实现了基于音频信号的手势识别系统，使用自建的扬声器麦克风平台 收集真实的音频数据对手势识别的准确率进行测试并评估系统性能，最终系统 的手势识别准确率为 98%。

The application of human sensing and gesture recognition in smart home has an urgent need and commercial value. In recent years, human-computer interaction with reasonable design and low additional hardware cost is a research priority. The existing work mainly bases on normal cameras, depth cameras and wearable sensors. The camera-based methods have been widely employed by technology companies. But the disadvantage of the camera-based methods is that it cannot be used in private occasion and is easily blocked by environmental objects. Meanwhile, camera-based methods are provided with higher energy consumption. The wearable devices-based methods have higher accuracy and low energy consumption, but consumers must purchase additional hardware. In this work, we investigated the principles of acoustic signal based gesture recognition system. We proposed a device-free gesture recognition system that can accurately sense the hand in-air movements using channel impulse response (CIR) and domain adaptation network. The CIR-based methods increase the measurement accuracy compared with the pioneer acoustic-based human gesture recognition system. To solve the problem that a gesture recognition model trained on a specific object does not work well when being applied to predict another object’s gestures, we proposed a domain adaptation network which can remove the environment information and object-specific features from the training data and increase the recognition accuracy. The main content of this thesis includes the following aspects: Firstly, we analyzed the features which are demodulated from different transmitted audio signals and designed different gesture recognition systems. We proposed a Channel Impulse Response based method which increases the accuracy for gesture detection. Secondly, we proposed a domain adaptation deep learning network which can increase the recognition accuracy to a higher level than the performance of traditional convolutional neural networks (CNN) based model. Finally, we implemented an acoustic based gesture recognition system. After that we simulated the algorithms and collected the real-word data to test our system. The feasibility of the gesture recognition system and the novel learning model were verified. The average recognition accuracy reached 98%.

信道估计 手势识别 领域自适应 深度学习 卷积神经网络 音频信号

channel impulse response gesture recognition domain adaptation deep-learning convolutional neural networks acoustic

中文

联合培养

南方科技大学

马鸿. 基于音频的人体手势识别系统设计与实现[D]. 深圳. 哈尔滨工业大学,2020.