面向智能人机交互的自然手势识别方法与设备研制

自然手势作为一种交互方式，在日常生活的交流中扮演着重要的角色，而且人体手势的敏捷、自然、高效等特点使其在人机交互任务中拥有广阔的应用前景。虽然应用手势的人机交互研究已经具有很长时间的发展并取得了一定的成果，但目前手势运动信息采集困难、传感器体积较大与人体相容性差、需要借助外部传感器等问题，限制了手势在人机交互任务中的实际应用。因此如何方便、高效的采集手势信息并让机器准确理解手势含义成为迫切需要解决的问题。
基于近年来新兴的柔性电子技术，本文首先对与人体皮肤相容的肌电传感器进行研究，并研制了一套柔性穿戴式手势控制设备，最终实现肌电信号采集及自然手势建模。本文的主要工作如下：
（1）提出了一种与人体皮肤相容的柔性可拉伸高密度肌电传感器，并进行机械和电学性能测试。本文以与人体皮肤相容的PDMS为基底、蛇形导电布为
导线、柔性可拉伸的银纳米线材料为电极制作肌电传感器。该传感器采用两层结构，内层为电极层，直接与皮肤接触采集信号；外层为导线层，将电极采集到信号传输到采集设备。经测试，该肌电传感器在循环弯曲和循环拉伸等条件下均保持了电阻相对稳定而没有被破坏，具有很高的机械稳定性，同时具有较低的电极-皮肤接触阻抗、较高的信噪比等电学特性。
（2）研制了包含采集电路和上、下位机软件的高密度肌电信号采集系统。由于肌电信号微弱，采用差分电极的方式采集肌电信号以提高抗干扰能力。上下位机通过无线局域网进行信息交互，下位机采集电路作为客户端能够实现64通道的差分肌电信号采集，上位机作为服务端完成数据的解析滤波显示保存等任务。
（3）应用自研设备实现手掌手势识别并搭建人-机交互实验平台。首先对比传统机器学习算法与深度学习算法在基于表面肌电信号的手势建模与识别中的异同，然后以自制的高密度肌电信号采集系统为基础，应用ANN模型实现手掌手势建模与识别，最后搭建人-无人履带车、无人飞机手势交互试验平台。

Handgestures, as an interactive way,playan important role in daily communication.With the character of agility, naturalness and high efficiency, hand gestures have been used more and more widelyinhuman-machine interaction.
Althoughthe human-machineinteraction researchesof gestures havebeen developed for a long time and havebeen made certain progress, there are still some problems and difficulties.The gesture motion information is difficult to collect,and the sensor is so biganduncomfortable, even it is necessary to use external sensors.These drawbacksmentionedlimit the further application of hand gestures in human-machine interaction,sohow toconvenientlycollectgesture informationand let machine understandsthe meaning of gestures becomesan urgent problem to solve.
Based on the emerging electronic skin, thisdissertationstarts from theresearch of high-density and stretchable sEMG sensor,thenfocusesonwearableequipment for recognition of nature gestures,andfinallycompleteshandgesture recognition.The main results and achievements are as follows:
Firstly, asensorthat collects surface EMG signalswas fabricatedwithflexible andstretchable silver nanowire material, serpentine wireand softPDMS. Thesensor is made in sandwich structure. The upper layer is a wire layer connecting to acquisition equipment,whilethe lower layer is an electrode layer directly contacting tothe skin. The EMG sensor has lower contact impedance and higher signal-to-noise ratio, anditsresistance is stable undermore than one thousandtests of cyclic bending and cyclic stretching.
Secondly,ahigh-densitysEMGacquisition systemis developed,including circuit and software.AssEMGsignal issoweak, a different electrode is used to collect the signal to improve the anti-interference ability. Theprincipal computer and itsslave realize communicationthrough the wireless LAN.Theslavecan achievethe 64 channelsdifferential electromyography signal acquisition, and the upper completesanalysis, filter and storagedata.
Finally,Palm gesture recognitioniscarried outwith self-developed devices.Westudythedifferencebetweentraditional machine learning and deep learningin hand gesture recognition based onmulti-channel sEMGsignaland thenwe use the ANN modelachievehand-gestures recognition.

[1]Army U S. The US Army Robotic and Autonomous Systems Strategy[J].FortEustis: TRADOC,2017:2-11.
[2]Yoon H S, Soh J, Bae Y J, et al. HandGestureRecognitionUsingCombinedFeatures ofLocation,Angle andVelocity[J]. Pattern Recognition,2001,34:1491-1501.
[3]Mitra S, Acharya T. GestureRecognition: A survey.IEEETransactions onSystems,Man & Cybernetics-PartC[C], 2007, 37(3):311-324
[4]Elmezain M, Al-Hamadi A, Michaelis B. HandTrajectory-basedGestureSpotting andRecognitionUsing HMM.IEEEInternationalConference onImageProcessing[C].Cairo, Egypt, 2009.3541-3544.
[5]Kaiman R E. ANewApproach toLinearFiltering andPredictionProblems[J].IEEE Assp Magazine, 1960,82(1):35-45.
[6]Higgins W T. AComparison ofComplementary and KalmanFiltering[J]. IEEETransactions on Aerospace and Electronic Systems, 1975 (3): 321-325.
[7]Xsens Technologies. Data sheet MTi 1-series[EB/OL].
[2018-06-20].https://www.farnell.com/datasheets/1937602.pdf.
[8]Junker H, Amft O, Lukowicz P, et al. GestureSpotting withBody-wornInertialSensors toDetectUserActivities[J]. Pattern Recognition, 2008, 41(6):2010–2024.
[9]Ramamoorthy A, Vaswani N, Chaudhury S, et al. Recognition ofDynamicHandGestures[J]. Pattern Recognition, 2003, 36(9): 2069-2081.
[10]Konrad P. The abc ofEMG:APracticalIntroduction toKinesiologicalElectromyography[J], 2005, 1(2005):30-35.
[11]杨义勇,王人成,贾晓红,等.人体肘关节复合运动的建模及协调控制[J].清华大学学报:自然科学版, 2004,44(5):653-656.
[12]颜志国.基于粗糙集和支持向量机的表面肌电特征约简和分类研究[D].上海:上海交通大学, 2008.25-38.
[13]ThalmicLabs. MYO臂环[EB/OL].https://www.thalmic.com/myo/.
[14]Assad C, Wolf M, Theodoridis T, et al. BioSleeve:ANatural EMG-basedInterface for HRI[C]//Proceedings of the 8th ACM/IEEE internationalconference on Human-robot interaction. IEEE Press, 2013: 69-70.
[15]Wolf M T, Assad C, Stoica A, et al. DecodingStatic andDynamicArm andHandGestures from the JPL BioSleeve[C]//2013 IEEE Aerospace Conference.IEEE, 2013: 1-9.
[16]Xiong A, Chen Y, Zhao X, et al. ANovel HCIBased on EMG andIMU[C]//2011 IEEE International Conference on Robotics and Biomimetics.IEEE, 2011: 2653-2657.
[17]Chen X, Zhang X, Zhao Z Y, et al. MultipleHandGestureRecognitionBasedonSurface EMGSignal[C]//2007 1st International conference onBioinformatics and Biomedical Engineering. IEEE, 2007: 506-509.
[18]Zhang X, Chen X, Wang W, et al. HandGestureRecognition andVirtualGameControlBased on 3DAccelerometer and EMGSensors[C]//Proceedings of the14th international conference on Intelligent user interfaces. ACM, 2009:401-406.
[19]ZhangX, Chen X, Li Y, et al. AFramework forHandGestureRecognitionBased onAccelerometer and EMGSensors[J]. IEEE Transactions on Systems,Man, and Cybernetics-Part A: Systems and Humans, 2011, 41(6): 1064-1076.
[20]Geng W, Du Y, Jin W, et al. GestureRecognition byInstantaneousSurfaceEMGImages[J]. Scientific reports, 2016, 6: 36571.
[21]Du Y, Jin W, Wei W, et al. Surface EMG-basedInter-sessionGestureRecognitionEnhanced byDeepDomainAdaptation[J]. Sensors, 2017, 17(3):458.
[22]Kim D H, Lu N, Ma R, et al. EpidermalElectronics[J]. science, 2011,333(6044): 838-843.
[23]Kim D H, Ghaffari R, Lu N, et al. Flexible andStretchableElectronics forBio-integratedDevices[J]. Annual review of biomedical engineering, 2012,14:113-128.
[24]Kabiri Ameri S, Ho R, Jang H, et al.GrapheneElectronicTattooSensors[J].ACS nano, 2017, 11(8): 7634-7641.
[25]Higgins W T. AComparison ofComplementary and KalmanFiltering[J]. IEEETransactions on Aerospace and Electronic Systems, 1975 (3): 321-325.
[26]Zhou Y, Wang Y, Liu R, et al. MultichannelNoninvasiveHuman–machineInterface viaStretchable µm thickSEMGPatches forRobotManipulation[J].Journal of Micromechanics and Microengineering, 2017, 28(1): 014005.
[27]Teachmeanatomyinfo. Muscles in thePosteriorCompartment of theForearm[EB/OL].
[2017-04-07].http://teachmeanatomy.info/upper-limb/muscles/posterior-forearm/.
[28]Teachmeanatomyinfo. Muscles in theAnteriorCompartment of theForearm[EB/OL].
[2017-04-07].http://teachmeanatomy.info/upper-limb/muscles/anterior-forearm/.29]许杰,贾旺,李伟,等.银纳米线制备及应用研究进展[J].化工新型材料,2018 (1): 9.
[30]Choi S B, Oh M S, Han C J, et al. Conformable, Thin, and Dry Electrode forElectrocardiography Using Composite of Silver Nanowires and PolyvinylButyral[J]. Electronic Materials Letters, 2019: 1-11.
[31]Dickey M D, Chiechi R C, Larsen R J, et al. Eutectic Gallium-Indium (EGaIn):A Liquid Metal Alloy for the Formation of Stable Structures in Microchannelsat Room Temperature[J]. Advanced Functional Materials, 2010,18(7):1097-1104.
[32]Wang J. Carbon‐nanotubeBasedElectrochemicalBiosensors: A review[J].Electroanalysis: An International Journal Devoted to Fundamental and PracticalAspects of Electroanalysis, 2005, 17(1): 7-14.
[33]赵章琰.表面肌电信号检测和处理中若干关键技术研究[D].合肥:中国科学技术大学, 2010: 20-55.
[34]Dunshee W K, Jacobson R L. AdhesiveBandage: U.S.Patent Application29/125,666[P]. 2002-3-26.
[35]唐天同,王兆宏.微纳加工科学原理[M].北京:电子工业出版社，2010:236-289.
[36]洪颖.仿变色龙舌头软体抓手的研究[D].哈尔滨：哈尔滨工业大学, 2017:33-35.
[37]Neuman M R. BiopotentialElectrodes[J]. MedicalInstrumentation:ApplicationandDesign, 1998, 3: 745-757.
[38]RosellJ, Colominas J, Riu P, et al. SkinImpedance from 1 Hz to 1 MHz[J].IEEE Transactions on Biomedical Engineering, 1988, 35(8): 649-651.
[39]Michael R Neuman. BiopotentialElectrodes[M].Medical instrumentation(4thedition),USA: John Wiley & Sons,2010:189-240
[40]Taheri B A, Knight R T, Smith R L. ADryElectrode for EEGRecording[J].Electroencephalography andClinicalNeurophysiology, 1994, 90(5): 376-383.
[41]MA300EMGSystemUserGuide[EB/OL].
[2018-10-1].https://www.motion-labs.com/pdf/ma300 user_manual.pdf.
[42]杨玉星.生物医学传感器与检测技术[M].北京：化学工业出版社，2005:318-321,344-346.
[43]Amma C, Krings T, Böer J, et al. AdvancingMuscle-computerInterfaces withHigh-densityElectromyography[C]//Proceedings of the 33rd Annual ACMConference on Human Factors in Computing Systems. ACM, 2015: 929-938.
[44]Reaz M B I, Hussain M S, Mohd-Yasin F. Techniques of EMGSignalAnalysis:Detection,Processing,Classification andApplications[J]. Biological procedures online, 2006, 8(1): 11.
[45]Texas Instruments, Inc. Ads129x用于生理信号测量的低功耗、8通道、24位模拟前端(rev k)[EB/OL].
[2018-02-22]. http://www.ti.com/cn/lit/gpn/ads1298.
[46]意法半导体(中国)投资有限公司. STM32F373C8[EB/OL].
[2018-02-22].http://www.st. com/resource/en/datasheet/stm32f373c8.pdf.
[47]Texas Instruments, Inc. Tps793Low-noise,High psrr, rf, 200-maLow-dropoutLinearRegulators inNanostarTMWaferChipScale and sot-23[EB/OL].
[2018-02-22]. http://www.ti.com/cn/lit/ gpn/tps793.
[48]Schwartz M. Internet of Things with ESP8266[M]. Packt Publishing Ltd,2016:128-145.
[49]The Qt Company. Qt[EB/OL].
[2017-05-01].https://www.qt.io/.
[50]Stevens W R. TCP/IPIllustrated vol. I: theProtocols[M]. Pearson EducationIndia, 1994.
[51]Sebelius F, Axelsson M, Danielsen N, et al. Real-timeControl of aVirtualHand[J]. Technology and Disability, 2005, 17(3): 131-141.
[52]卫文韬.肌电手势识别中的多流融合和多视图深度学习方法研究[D].杭州:浙江大学, 2018:28-45.
[53]杜宇.基于深度机器学习的体态与手势感知计算关键技术研究[D].杭州:浙江大学, 2017: 36-50.
[54]Bishop C M. PatternRecognition andMachineLearning[M].Springer,2006:160-202.
[55]LeCun Y, Bengio Y, Hinton G. Deep learning[J]. nature, 2015, 521(7553): 436.
[56]Proakis J G. DigitalSignalProcessing:PrinciplesAlgorithms andApplications[M]. Pearson Education India, 2001: 325-353.
[57]J. O. Smith, Introduction to Digital Filters with Audio Applications[M], W3KPublishing,2007: 212-230.
[58]Zhang X, Chen X, Zhao Z, et al. Research onGestureDefinition andElectrodePlacement inPatternRecognition ofHandGestureActionSEMG[C]//International Conference on Medical Biometrics. Springer,Berlin,Heidelberg, 2008: 33-40.
[59]Géron A. Hands-onMachineLearning with Scikit-Learn and TensorFlow:Concepts,Tools, andTechniques toBuildIntelligentSystems[M]. " O'ReillyMedia, Inc.", 2017: 341-356.
[60]TensorFlow中文社区[EB/OL].
[2019-2-26].https://www.tensorflow.org/.