基于柔性导光材料的一体式可穿戴手套及手势识别研究

可穿戴设备技术作为人机交互技术中最具前景的一部分，近年来已经得到了飞速发展。其中，基于可穿戴设备的手势识别有着广阔的应用前景，例如手语翻译、医疗康复和虚拟现实交互等。相比于视觉方案，基于可穿戴设备的手势识别能够克服照明条件、视野遮挡和算力限制等缺点，但目前多数相关研究存在传感器数量难以匹配手部多个自由度的问题。因此相近手势的识别率较低。此外，在算法方面，以往常用的时序网络对于静态手势并不完全适用，需要新的网络模型来解决这个问题。

本文针对可穿戴手套的硬件和算法进行了相关研究。在硬件方面，本文设计了一款体积极小的光电式弯曲传感器。该传感器以焊有发光二极管和光敏二极管的电路板为单元，单元间使用导光材料作为光路。当手指弯曲带动光路产生变化，光敏二极管所接收的光强也随之变化，从而实现手指的弯曲检测。线性序列扫描实现了多个单元间的元件复用。针对柔性导光材料本文也进行了选材与实验。后续将多个传感器布置在全手手指关节两侧位置制作了可穿戴手套，实现了对手部的15个关节弯曲信号的采集。基于制作的可穿戴手套采集了手势数据集，并使用数据降维验证了数据集的质量。在算法方面，本文基于深度学习，利用图卷积网络的空间性，将传感器的空间位置信息通过图卷积网络加入到算法中。实验结果显示，将提出的算法与时序网络结合组成时空网络，能够实现24个静态字母手势的高准确率识别。

[1] 黄慎泽, 王俊贤. 人机交互技术在虚拟偶像中的应用研究[J]. 传媒, 2021(16): 51-53.
[2] CASALE P, PUJOL O, RADEVA P. Human activity recognition from accelerometer data using a wearable device[C]//Iberian Conference on Pattern Recognition and Image Analysis. Springer, Berlin, Heidelberg, 2011: 289-296.
[3] ELKOURA G, SINGH K. Handrix: animating the human hand[C]//Proceedings of the 2003 ACM SIGGRAPH/Eurographics Symposium on Computer Animation. 2003: 110-119
[4] OUDAH M, AL-NAJI A, CHAHL J. Hand gesture recognition based on computer vision: a review of techniques[J]. Journal of Imaging, 2020, 6(8): 73.
[5] 井云鹏, 范基胤, 王亚男, 等. 智能传感器的应用与发展趋势展望[J]. 黑龙江科技信息, 2013(21): 111-112.
[6] 李森. 离电柔性压力传感技术及其应用研究[D]. 合肥: 中国科学技术大学, 2020.
[7] 刘岚馨. 基于可穿戴设备的多传感器手势设计与识别方法研究[D]. 西安: 西安电子科技大学, 2019.
[8] YANG J CMUN J, KWON S Y, et al. Electronic skin: recent progress and future prospects for skin‐attachable devices for health monitoring, robotics, and prosthetics[J]. Advanced Materials, 2019, 31(48): 1904765.
[9] 任海兵, 祝远, 徐光祐, 等. 基于视觉手势识别的研究-综述[J]. 电子学报, 2000: 118-121.
[10] ZHOU Z, CHEN K, LI X, et al. Sign-to-speech translation using machine-learning-assisted stretchable sensor arrays[J]. Nature Electronics, 2020, 3(9): 571-578.
[11] SHAHMIRI F, DIETZ P H. Sharc: A geometric technique for multi-bend/shape sensing[C]//Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 2020: 1-12.
[12] TAKADA R, KADOMOTO J, SHIZUKI B. A sensing technique for data glove using conductive fiber[C]//Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. 2019: 1-4.
[13] DELPRETO J, HUGHES J, D'ARIA M, et al. A wearable smart glove and its application of pose and gesture detection to sign language classification[J]. IEEE Robotics and Automation Letters, 2022, 7(4): 10589-10596.
[14] KIM S, JEONG D, OH J, et al. A novel all-in-one manufacturing process for a soft sensor system and its application to a soft sensing glove[C]//2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2018: 7004-7009.
[15] BAI H, LI S, BARREIROS J, et al. Stretchable distributed fiber-optic sensors[J]. Science, 2020, 370(6518): 848-852.
[16] JEONG E, LEE J, KIM D E. Finger-gesture recognition glove using velostat (ICCAS 2011)[C]//2011 11th International Conference on Control, Automation and Systems. IEEE, 2011: 206-210.
[17] 张婷. 基于压电陶瓷的钢管混凝土柱界面剥离损伤监测的实验研究[D]. 长沙: 湖南大学, 2011.
[18] 韩若冰. 多晶LuFe2O4的制备及其介电性能研究[D]. 成都: 电子科技大学, 2012.
[19] SYU M H, GUAN Y J, LO W C, et al. Biomimetic and porous nanofiber-based hybrid sensor for multifunctional pressure sensing and human gesture identification via deep learning method[J]. Nano Energy, 2020, 76: 105029.
[20] SUNDARAM S, KELLNHOFER P, LI Y, et al. Learning the signatures of the human grasp using a scalable tactile glove[J]. Nature, 2019, 569(7758): 698-702.
[21] WANG Z L, SONG J. Piezoelectric nanogenerators based on zinc oxide nanowire arrays[J]. Science, 2006, 312(5771): 242-246.
[22] WEN F, ZHANG Z, HE T, et al. AI enabled sign language recognition and VR space bidirectional communication using triboelectric smart glove[J]. Nature Communications, 2021, 12(1): 5378.
[23] LUO Y, WANG Z, WANG J, et al. Triboelectric bending sensor based smart glove towards intuitive multi-dimensional human-machine interfaces[J]. Nano Energy, 2021, 89: 106330.
[24] MITRA S, ACHARYA T. Gesture recognition: A survey[J]. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 2007, 37(3): 311-324.
[25] FANG B, GUO D, SUN F, et al. A robotic hand-arm teleoperation system using human arm/hand with a novel data glove[C]//2015 IEEE International Conference on Robotics and Biomimetics (ROBIO). IEEE, 2015: 2483-2488.
[26] WANG Y, MA H. Real-time continuous gesture recognition with wireless wearable imu sensors[C]//2018 IEEE 20th International Conference on e-Health Networking, Applications and Services (Healthcom), Ostrava, Czech Republic, 2018:1-6.
[27] HOCHREITER S, SCHMIDHUBER J. Long short-term memory[J]. Neural Computation, 1997, 9(8): 1735-1780.
[28] 陆继翔, 张琪培, 杨志宏, 等. 基于CNN-LSTM混合神经网络模型的短期负荷预测方法[J]. 电力系统自动化, 2019, 43(08): 131-137.
[29] TSINGANOS P, CORNELIS B, CORNELIS J, et al. Improved gesture recognition based on sEMG signals and TCN[C]//ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2019: 1169-1173.
[30] BAI S, KOLTER J Z, KOLTUN V. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling[J]. arXiv preprint arXiv:1803.01271, 2018.
[31] TANG Y, TIAN Y, LU J, et al. Deep progressive reinforcement learning for skeleton-based action recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018: 5323-5332.
[32] THAKKAR K, NARAYANAN P J. Part-based graph convolutional network for action recognition[J]. arXiv preprint arXiv:1809.04983, 2018.
[33] SHI L, ZHANG Y, CHENG J, et al. Skeleton-based action recognition with directed graph neural networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019: 7912-7921.
[34] SI C, CHEN W, WANG W, et al. An attention enhanced graph convolutional lstm network for skeleton-based action recognition[C]//proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019: 1227-1236.
[35] 韩璐斌. 光电式传感器的原理及其特性[J]. 科技创新与应用, 2020, No.302(10): 77-78.
[36] 洪佩智. 全反射现象及其在集成光学中的应用[J]. 物理, 1985(07): 429-434.
[37] 任雪光, 宁传刚, 邓景康, 等. 硅PIN光敏二极管探测X、γ射线的性能及应用[J].核电子学与探测技术, 2003(01): 30-32+29.
[38] 胡先志, 支敬敬, 胡爱武. 光纤材料的研究进展[J]. 光通信技术, 1996(01): 66-69.
[39] 张培培. 稀土纳米粒子/PMMA复合聚合物的制备及三维显示应用[D].长春: 吉林大学.
[40] 王明英, 王健, 杜思辰, 等.硅橡胶硫化体系研究进展[J]. 合成橡胶工业, 2022, 45(03): 244-252.
[41] 王先淋, 张正, 王文元, 等. 一种PMMA超薄超透薄膜及其制作方法: 中国, CN107903556A[P]. 2020-06-02.
[42] 邱剑, 吕世豪, 玛丽娅, 等. 电子辐照PDMS力学性能影响实验研究[J]. 应用力学学报, 2022, 39(06): 1007-1013.
[43] AMJADI M, YOON Y J, PARK I. Ultra-stretchable and skin-mountable strain sensors using carbon nanotubes–Ecoflex nanocomposites[J]. Nanotechnology, 2015, 26(37): 375501.
[44] MAXWELL R S, BALAZS B, COHENOUR R, et al. Aging of silica‐filled PDMS/PDPS copolymers in desiccating environments: a DSC and NMR study[C]//Macromolecular Symposia. Weinheim: WILEY‐VCH Verlag, 2003, 202(1): 291-296.
[45] PARK S, MONDAL K, TREADWAY III R M, et al. Silicones for stretchable and durable soft devices: Beyond Sylgard-184[J]. ACS Applied Materials & Interfaces, 2018, 10(13): 11261-11268.
[46] 张文华. 柔性防水材料的蠕变性及其实验研究[C]//中国建筑学会防水技术专业委员会,中国防水技术网.防水工程与材料学术论坛论文集.中国建筑学会施工与建材分会防水技术专业委员会, 2010: 19-21.
[47] 陶俐言, 蒋昕洁. 人手抓取运动轨迹研究[J]. 机械科学与技术, 2017, 36(09): 1345-1350.
[48] 刘博, 张玉茹, 任大伟, 等. 人手食指运动学建模[J]. 机器人, 2007(03): 214-218.
[49] 彭秀丽, 李全忠, 王延年. 基于长短期记忆网络与门循环单元模型的长时程低血糖预警研究[J]. 医药论坛杂志, 2021, 42(11):5.
[50] VAN DER MAATEN L, HINTON G. Visualizing data using t-SNE[J]. Journal of Machine Learning Research, 2008, 9(11).
[51] Kipf T N, Welling M. Semi-supervised classification with graph convolutional networks[J]. arXiv preprint arXiv:1609.02907, 2016.
[52] GARCIA-GARCIA A, ZAPATA-IMPATA B S, ORTS-ESCOLANO S, et al. Tactilegcn: A graph convolutional network for predicting grasp stability with tactile sensors[C]//2019 International Joint Conference on Neural Networks (IJCNN). IEEE, 2019: 1-8.
[53] WU Z, PAN S, CHEN F, et al. A comprehensive survey on graph neural networks[J]. IEEE Transactions on Neural Networks and Learning Systems, 2020, 32(1): 4-24.
[54] HUANG B, XIONG C, CHEN W, et al. Common kinematic synergies of various human locomotor behaviours[J]. Royal Society Open Science, 2021, 8(4): 210161.
[55] LI M, CHEN S, CHEN X, et al. Actional-structural graph convolutional networks for skeleton-based action recognition[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019: 3595-3603.
[56] ZHANG P, LAN C, ZENG W, et al. Semantics-guided neural networks for efficient skeleton-based human action recognition[C]//proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2020: 1112-1121.
[57] 孙鑫, 李振华, 董军宇, 等. 面向海洋数据的复杂网络建模及可视化分析[J]. 系统仿真学报, 2018, 30(07): 2445-2452.
[58] 张蔚澜, 齐华, 李胜. 时空图卷积网络在人体异常行为识别中的应用[J]. 计算机工程与应用, 2022, 58(12): 122-131.
[59] HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016: 770-778.