中文版 | English
题名

基于 Ecoflex 的柔性光电式传感器及手势识别研究

其他题名
Flexible photoelectric sensor based on Ecoflex and gesture recognition research
姓名
姓名拼音
YE Chaoxiang
学号
12032269
学位类型
硕士
学位专业
0856 材料与化工
学科门类/专业学位类别
0856 材料与化工
导师
易正琨
导师单位
中国科学院深圳先进技术研究院
论文答辩日期
2022-05-11
论文提交日期
2022-06-25
学位授予单位
南方科技大学
学位授予地点
深圳
摘要
  随着人机交互技术的发展,手势识别广泛应用于虚拟现实、医疗、服务、工业等领域。手势不仅在游戏娱乐中可以为玩家带来真实的体验感,还能够提高人们的生活质量,为聋哑人带来福音。目前最主要的手势识别方法有两类,一是基于视觉可穿戴设备的手势识别,缺点是视觉在实际应用中容易受到照明条件和视野遮挡等外界因素的影响,很难实现高精度的手势识别。二是基于触觉可穿戴设备的手势识别,虽然使用触觉传感能够避免外界因素的干扰,但是现有的触觉传感器通常只能识别一维空间的手势运动,难以完成复杂的手势识别任务。
  为了解决以上问题,本文设计了基于触觉可穿戴设备和深度学习的手势识别系统,首先设计并制作了基于 Ecoflex 的光电式传感器及其可穿戴设备,基于光强调制的原理,使用不同位置分布的光敏元件赋予传感器感受二维空间力学刺激的能力,实现了全方向弯曲识别。接着设计并搭建了硬件电路系统和信号采集系统,完成了美国手势语言数据集的构建,该数据集包含 24 个静态的英文字母手势,一共 36000 帧数据,并进行了数据预处理,为触觉手势识别领域贡献了宝贵的数据资源。最后在 TASLD 数据集上
使用传统机器学习算法和深度学习算法验证了所提出的手势识别系统的识别性能,并基于正则化策略,提出了两种适用于触觉手势识别任务的深度学习算法,与基准方法相比具有更好的识别准确率和鲁棒性,实现了对复杂手势任务的高精度识别。
关键词
语种
中文
培养类别
独立培养
入学年份
2020
学位授予年份
2022-05
参考文献列表

[1] 王苏振.基于深度学习的手势识别技术研究[D].杭州:浙江大学,2019.
[2] Hsien-Ting Chang, Jen-Yuan Chang. Sensor Glove Based on Novel Inertial Sensor Fusion Control Algorithm for 3-D Real-Time Hand Gestures Measurements[J]. IEEE Transactions on Industrial Electronics, 2020, 67(1):658-666.
[3] Wang Y T, Ma H P. Real-Time Continuous Gesture Recognition with Wireless Wearable IMU Sensors[C]//2018 IEEE 20th International Conference on e -Health Networking, Applications and Services (Healthcom). IEEE, 2018.
[4] Jian W , Lu S , Jafari R . A Wearable System for Recognizing American Sign Language in Real-Time Using IMU and Surface EMG Sensors[J]. IEEE Journal of Biomedical and Health Informatics, 2016, 20(5):1281-1290.
[5] Lederman S J, Klatzky R L. Extracting object properties through haptic exploration[J]. Acta Psychol, 1993, 84(1):29-40.
[6] Wen F, Sun Z, He T, et al. Machine learning glove using self‐powered conductive superhydrophobic triboelectric textile for gesture recognition in VR/AR applications[J]. Advanced Science, 2020, 7(14): 2000261.
[7] Zhu M, Sun Z, Zhang Z, et al. Haptic-feedback smart glove as a creative human machine interface (HMI) for virtual/augmented reality applications[J]. Science Advances, 2020, 6(19): eaaz8693.
[8] Zhou Z, Chen K, Li X, et al. Sign-to-speech translation using machine-learning assisted stretchable sensor arrays[J]. Nature Electronics, 2020, 3(9): 571 -578.
[9] Yuan Y, Liu Y, Barner K. Tactile gesture recognition for people with disabilities[C]//Proceedings.(ICASSP'05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005. IEEE, 2005, 5: v/461-v/464 Vol. 5.
[10] Huang J , Yang X , Yu J, et al. A Universal and Arbitrary Tactile Interactive System Based on Self-powered Optical Communication[J]. Nano Energy, 2019, 69:104419.
[11] Freeman E, Brewster S, Lantz V. Towards in-air gesture control of household appliances with limited displays[C]//IFIP Conference on Human-Computer Interaction. Springer, Cham, 2015: 611-615.
[12] Fruchard B, Lecolinet E, Chapuis O. Side-Crossing Menus: Enabling Large Sets of Gestures for Small Surfaces[J]. Proceedings of the ACM on Human-Computer Interaction, 2020, 4(ISS): 1-19.
[13] Pusch A, Noël F. Augmented reality for operator training on industrial workplaces-Comparing the Microsoft hololens vs. small and big screen tactile devices[C]//IFIP International Conference on Product Lifecycle Management. Springer, Cham, 2019: 3-13.
[14] de Gea Fernández J, Mronga D, Günther M, et al. Multimodal sensor-based wholebody control for human–robot collaboration in industrial settings[J]. Robotics and Autonomous Systems, 2017, 94: 102-119.
[15] 刘宇昕.智能传感器的应用与发展趋势展望[J].数码设计(下),2019, 000(002):206.
[16] Lipomi D J, Vosgueritchian M, Tee B C K, et al. Skin-like pressure and strain sensors based on transparent elastic films of carbon nanotubes[J]. Nature Nanotechnology, 2011, 6(12): 788-792.
[17] Wang X, Dong L, Zhang H, et al. Recent progress in electronic skin[J]. Advanced Science, 2015, 2(10): 1500169.
[18] Yang J C, Mun J, Kwon S Y, et al. Electronic skin: recent progress and future prospects for skin ‐ attachable devices for health monitoring, robotics, and prosthetics[J]. Advanced Materials, 2019, 31(48): 1904765.
[19] 任 海 兵 , 祝 远 新 , 徐 光 , 等 . 基 于 视 觉 手 势 识 别 的 研 究 — 综 述 [J]. 电子学报,2000,28(2):118-121.
[20] Luo N, Dai W, Li C, et al. Flexible piezoresistive sensor patch enabling ultralow power cuffless blood pressure measurement[J]. Advanced Functional Materials, 2016, 26(8): 1178-1187.
[21] Zhu S E, Krishna Ghatkesar M, Zhang C, et al. Graphene based piezoresistive pressure sensor[J]. Applied Physics Letters, 2013, 102(16): 161904.
[22] Sundaram S, Kellnhofer P, Li Y, et al. Learning the signatures of the human grasp using a scalable tactile glove[J]. Nature, 2019, 569(7758): 698 -702.
[23] Jiang S, Li L, Xu H, et al. Stretchable e-skin patch for gesture recognition on the back of the hand[J]. IEEE Transactions on Industrial Electronics, 2019, 67(1): 647-657.
[24] Esposito D, Andreozzi E, Gargiulo G D, et al. A piezoresistive array armband with reduced number of sensors for hand gesture recognition[J]. Frontiers in Neurorobotics, 2020: 114.
[25] Yao S, Zhu Y. Wearable multifunctional sensors using printed stretchable conductors made of silver nanowires[J]. Nanoscale, 2014, 6(4): 2345-2352.
[26] Wong R D P, Posner J D, Santos V J. Flexible microfluidic normal force sensor skin for tactile feedback[J]. Sensors and Actuators A: Physical, 2012, 179: 62-69.
[27] Shahmiri F, Dietz P H. Sharc: A geometric technique for multi-bend/shape sensing[C]//Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 2020: 1-12.
[28] Nie B, Li R, Cao J, et al. Flexible transparent iontronic film for interfacial capacitive pressure sensing[J]. Advanced Materials, 2015, 27(39): 6055 -6062.
[29] Nie B, Li R, Brandt J D, et al. Microfluidic tactile sensors for three -dimensional contact force measurements[J]. Lab on a Chip, 2014, 14(22): 4344 -4353.
[30] Li T, Luo H, Qin L, et al. Flexible capacitive tactile sensor based on micropatterned dielectric layer[J]. Small, 2016, 12(36): 5042-5048.
[31] Nie B, Li R, Brandt J D, et al. Iontronic microdroplet array for flexible ultrasensitive tactile sensing[J]. Lab on a Chip, 2014, 14(6): 1107-1116.
[32] Li R, Nie B, Zhai C, et al. Telemedical wearable sensing platform for management of chronic venous disorder[J]. Annals of Biomedical Engineering, 2016, 44(7): 2282-2291.
[33] 李森.离电柔性压力传感技术及其应用研究[D].安徽:中国科学技术大学,2020.
[34] Zhu Z, Li R, Pan T. EIS: a wearable device for epidermal pressure sensing[C]//2018 IEEE Haptics Symposium (HAPTICS). IEEE, 2018: 1-6.
[35] Park K I, Son J H, Hwang G T, et al. Highly‐efficient, flexible piezoelectric PZT thin film nanogenerator on plastic substrates[J]. Advanced Materials, 2014, 26(16): 2514-2520.
[36] Dagdeviren C, Su Y, Joe P, et al. Conformable amplified lead zirconate titanate sensors with enhanced piezoelectric response for cutaneous pressure monitoring[J]. Nature Communications, 2014, 5(1): 1-10.
[37] Li C, Wu P M, Lee S, et al. Flexible dome and bump shape piezoelectric tactile sensors using PVDF-TrFE copolymer[J]. Journal of Microelectromechanical Systems, 2008, 17(2): 334-341.
[38] Syu M H, Guan Y J, Lo W C, et al. Biomimetic and porous nanofiber-based hybrid sensor for multifunctional pressure sensing and human gesture identification via deep learning method[J]. Nano Energy, 2020, 76: 105029.
[39] Gao C, Long Z, Zhong T, et al. A self-powered intelligent glove for real-time human-machine gesture interaction based on piezoelectric effect of T-ZnO/PVDF film[J]. Journal of Physics D: Applied Physics, 2022, 55(19): 194004.
[40] Yan L, Mi Y, Lu Y, et al. Weaved piezoresistive triboelectric nanogenerator for human motion monitoring and gesture recognition[J]. Nano Energy, 2022, 96: 107135.
[41] Zhao H, O’Brien K, Li S, et al. Optoelectronically innervated soft prosthetic hand via stretchable optical waveguides[J]. Science Robotics, 2016, 1(1): eaai7529.
[42] Huang J, Zhou W, Li H, et al. Attention-based 3D-CNNs for large-vocabulary sign language recognition[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2018, 29(9): 2822-2832.
[43] Min Y, Zhang Y, Chai X, et al. An efficient pointlstm for point clouds based gesture recognition[C]//Proceedings of the IEEE/CVF Conference on Computer Visio n and Pattern Recognition. 2020: 5761-5770.
[44] Molchanov P, Yang X, Gupta S, et al. Online detection and classification of dynamic hand gestures with recurrent 3d convolutional neural network[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016: 4207-4215.
[45] Rodolà E, Cosmo L, Litany O, et al. SHREC'17: Deformable shape retrieval with missing parts[C]//10th Eurographics Workshop on 3D Object Retrieval, 3DOR 2017. Eurographics Association, 2017: 85-94.
[46] Wang H, Chai X, Chen X. A novel sign language recognition framework using hierarchical Grassmann covariance matrix[J]. IEEE Transactions on Multimedia, 2019, 21(11): 2806-2814.
[47] Huang X, Wang Q, Zang S, et al. Tracing the motion of finger joints for gesture recognition via sewing RGO-coated fibers onto a textile glove[J]. IEEE Sensors Journal, 2019, 19(20): 9504-9511.
[48] Liu G Y, Kong D Y, Hu S G, et al. Smart electronic skin having gesture recognition function by LSTM neural network[J]. Applied Physics Letters, 2018, 113(8): 084102.
[49] Yuan G, Liu X, Yan Q, et al. Hand gesture recognition using deep feature fusion network based on wearable sensors[J]. IEEE Sensors Journal, 2020, 21(1): 539 -547.
[50] Garcia-Garcia A, Zapata-Impata B S, Orts-Escolano S, et al. Tactilegcn: A graph convolutional network for predicting grasp stability with tactile sensors[C]//2019 International Joint Conference on Neural Networks (IJCNN). IEEE, 2019: 1 -8.
[51] Gao R, Taunyazov T, Lin Z, et al. Supervised autoencoder joint learning on heterogeneous tactile sensory data: Improving material classification performance[C]//2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2020: 10907-10913.
[52] Ryu S C, Quek Z F, Renaud P, et al. An optical actuation system and curvature sensor for a MR-compatible active needle[C]//2012 IEEE International Conference on Robotics and Automation. IEEE, 2012: 1589-1594.
[53] Dobrzynski M K, Halasz I, Pericet-Camara R, et al. Contactless deflection sensing of concave and convex shapes assisted by soft mirrors[C]//2012 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2012: 4810-4815.
[54] Polygerinos P, Ataollahi A, Schaeffter T, et al. MRI-compatible intensity-modulated force sensor for cardiac catheterization procedures[J]. IEEE Transactions on Biomedical Engineering, 2010, 58(3): 721-726.
[55] Polygerinos P, Seneviratne L D, Razavi R, et al. Triaxial catheter-tip force sensor for MRI-guided cardiac procedures[J]. IEEE/ASME Transactions on Mechatronics, 2012, 18(1): 386-396.
[56] Fainman Y, Lee L, Psaltis D, et al. Optofluidics: Fundamentals, Devices, and Applications[M]. McGraw Hill Professional, 2009.
[57] Pegan J D, Ho A Y, Bachman M, et al. Flexible shrink-induced high surface area electrodes for electrochemiluminescent sensing[J]. Lab on a Chip, 2013, 13(21): 4205-4209.
[58] Li Z, Cheng L, Song Q. An ultra-stretchable and highly sensitive photoelectric effect-based strain sensor: implementation and applications[J]. IEEE Sensors Journal, 2020, 21(4): 4365-4376.
[59] Kashi S, Gupta R K, Baum T, et al. Dielectric properties and electromagnetic interference shielding effectiveness of graphene-based biodegradable nanocomposites[J]. Materials & Design, 2016, 109: 68-78.
[60] Danilov V, Dölle C, Ott M, et al. Plasma treatment of polydimethylsiloxane thin films studied by infrared reflection absorption spectroscopy[J]. The 29th International Conference on Phenomena in Ionized Gases, 2009.
[61] Wu C Y, Liao W H, Tung Y C. Integrated ionic liquid-based electrofluidic circuits for pressure sensing within polydimethylsiloxane microfluidic systems[J]. Lab on a Chip, 2011, 11(10): 1740-1746.
[62] Liu T, Sen P, Kim C J. Characterization of nontoxic liquid -metal alloy galinstan for applications in microdevices[J]. Journal of Microelectromechanical Systems, 2011, 21(2): 443-450.
[63] Polygerinos P, Ataollahi A, Schaeffter T, et al. MRI-compatible intensity-modulated force sensor for cardiac catheterization procedures[J]. IEEE Transactions on biomedical engineering, 2010, 58(3): 721-726.
[64] Ozioko O, Dahiya R. Smart tactile gloves for haptic interaction, communication, and rehabilitation[J]. Advanced Intelligent Systems, 2022, 4(2): 2100091.
[65] Al-Handarish Y, Omisore O M, Chen J, et al. A Hybrid Microstructure Piezoresistive Sensor with Machine Learning Approach for Gesture Recognition[J]. Applied Sciences, 2021, 11(16): 7264.
[66] Yan L, Mi Y, Lu Y, et al. Weaved piezoresistive triboelectric nanogenerator for human motion monitoring and gesture recognition[J]. Nano Energy, 2022, 96: 107135.
[67] Pugeault N, Bowden R. Spelling it out: Real-time ASL fingerspelling recognition[C]//2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops). IEEE, 2011: 1114-1119.
[68] Van der Maaten L, Hinton G. Visualizing data using t-SNE[J]. Journal of Machine Learning Research, 2008, 9(11).
[69] Le L, Patterson A, White M. Supervised autoencoders: Improving generalization performance with unsupervised regularizers[J]. Advances in Neural Information Processing Systems, 2018, 31.

所在学位评定分委会
中国科学院深圳理工大学(筹)联合培养
国内图书分类号
TP391.4
来源库
人工提交
成果类型学位论文
条目标识符http://sustech.caswiz.com/handle/2SGJ60CL/342776
专题中国科学院深圳理工大学(筹)联合培养
推荐引用方式
GB/T 7714
叶超翔. 基于 Ecoflex 的柔性光电式传感器及手势识别研究[D]. 深圳. 南方科技大学,2022.
条目包含的文件
文件名称/大小 文献类型 版本类型 开放类型 使用许可 操作
12032269-叶超翔-中国科学院深圳(4282KB)----限制开放--请求全文
个性服务
原文链接
推荐该条目
保存到收藏夹
查看访问统计
导出为Endnote文件
导出为Excel格式
导出为Csv格式
Altmetrics Score
谷歌学术
谷歌学术中相似的文章
[叶超翔]的文章
百度学术
百度学术中相似的文章
[叶超翔]的文章
必应学术
必应学术中相似的文章
[叶超翔]的文章
相关权益政策
暂无数据
收藏/分享
所有评论 (0)
[发表评论/异议/意见]
暂无评论

除非特别说明,本系统中所有内容都受版权保护,并保留所有权利。