柔性自适应手指基于视觉的力触觉感知技术及其协作机器人应用研究

  随着机器人领域的发展，机器人与人进行物理交互已成为机器人不可避免的行为。在实现机器人系统与物理环境稳健交互的过程中，柔性力传感器扮演了重要的角色。柔性力传感器-机器人系统提高协作能力的同时也带来了诸多难题。其中主要的难点和关键的问题有：1.柔性体存在迟滞效应。2.建立柔性力传感器与机器人的交互控制模型。3.传统的基于碰撞检测的方法在使用柔性体的人机交互上并不适用。本文根据这些问题详细探讨柔性手指的力传感器模型，以及其在人机交互、碰撞检测等力控领域的广泛应用的问题。

  通过柔性手指参数辨识研究，全面探究各类手指动力学模型的力学特性和动态响应。采用非线性最小二乘法对手指进行弹簧-阻尼模型、N-DP模型和Bouc-wen模型的拟合分析，表明本文提出的N-DP模型在柔性手指力预测的精度和消除迟滞效应方面具有显著优势。

  深入研究了柔性手指-机器人系统的阻抗性能表现，特别是在人机交互、避障以及未知环境探索任务中的性能。建立了柔性手指-机器人系统的模型，分析系统的控制器参数及材料参数，以提高机器人在执行各种任务时的性能和稳定性。

  由于基于阈值的碰撞检测方法在柔性体上进行交互任务时检测碰撞的可行性差，本文探索了基于频域分析的柔性力传感器碰撞检测方法。通过利用柔性手指系统进行频域分析，进一步研究了柔性手指在人机协作中的性能表现，并对意外碰撞和主动交互行为的频域特征进行了比对分析。其可以使得柔性手指系统能够准确识别和区分这两种情况下的行为模式，有效地保障了人机协作的安全和稳定性。
  
  本文建立了一个交互实验平台，验证了柔性手指力传感器在机器人与环境互动中的有效性。实验结果显示柔性手指在互动任务中具有良好的适应性、实时响应性和区分意外碰撞与主动交互的能力。在焊接及轨迹跟踪实验中，针对可能出现的焊缝路径误差，我们提出了使用柔性手指系统微调焊枪位置的创新方法，以确保焊接过程中的准确对齐，并提前警示可能的障碍物，避免碰撞和损坏。实验结果清晰展示了柔性手指传感器在感知外部力和快速响应方面的准确性，以及在不同情境下的稳定性和可靠性。

  As the field of robotics advances, the physical interaction between robots and humans has become inevitable. In the process of achieving robust interaction between robot systems and the physical environment, flexible force sensors play an important role.

  Flexible force sensor-robot systems not only enhance collaboration but also bring about numerous challenges. The main difficulties and key issues include: 1. Hysteresis effect in flexible bodies. 2. Establishing an interaction control model between flexible force sensors and robots. 3. Traditional collision detection methods are not suitable for human-machine interaction using flexible bodies. This paper elaborates on these issues and discusses in detail the force sensor model of flexible fingers, as well as the widespread application of such sensors in force control fields such as human-machine interaction and collision detection.

  Through research on the identification of flexible finger parameters, various mechanical characteristics and dynamic responses of finger dynamic models are comprehensively explored. The spring-damping model, N-DP model, and Bouc-Wen model are fitted and analyzed using nonlinear least squares method, showing that the proposed N-DP model has significant advantages in the accuracy of flexible finger force prediction and hysteresis elimination.

  The impedance performance of the flexible finger-robot system is deeply researched, especially in tasks such as human-machine interaction, obstacle avoidance, and exploration of unknown environments. A model of the flexible finger-robot system is established to analyze the controller parameters and material parameters of the system, in order to improve the performance and stability of the robot in executing various tasks.

  Due to the poor feasibility of collision detection using threshold-based methods when interacting with flexible bodies, this paper explores a collision detection method based on frequency domain analysis of flexible force sensors. By utilizing frequency domain analysis with flexible finger systems, the performance of flexible fingers in human-machine cooperation is further studied, and comparative analysis of the frequency domain characteristics of accidental collisions and active interaction behaviors is conducted. This enables the flexible finger system to accurately identify and distinguish between the two behavioral patterns, effectively ensuring the safety and stability of human-machine cooperation.

  This paper establishes an interactive experimental platform to validate the effectiveness of flexible finger force sensors in robot-environment interaction. Experimental results demonstrate that flexible fingers exhibit good adaptability, real-time responsiveness, and the ability to distinguish between accidental collisions and active interaction during interactive tasks. In welding and trajectory tracking experiments, an innovative method of fine-tuning the welding gun position using flexible finger systems is proposed to ensure accurate alignment during welding and to provide early warnings of potential obstacles, thus avoiding collisions and damage. The experimental results clearly demonstrate the accuracy of flexible finger sensors in perceiving external forces and their rapid response, as well as their stability and reliability in different contexts.

[1] CORONADO E, SHINYA T, VENTURE G. Hold my hand: development of a force controller and system architecture for joint walking with a companion robot[J]. Sensors, 2023, 23(12): 5692.
[2] QIU S, PEI Z, WANG C, et al. Systematic review on wearable lower extremity robotic exoskeletons for assisted locomotion[J]. Journal of Bionic Engineering, 2023, 20(2): 436-469.
[3] SIRINTUNA D, GIAMMARINO A, AJOUDANI A. An object deformation-agnostic framework for human–robot collaborative transportation[J]. IEEE Transactions on Automation Science and Engineering, 2023.
[4] WANG Y H, LIU G Y, HUANG G, et al. Variable admittance force feedback device and its human-robot interaction stability[J]. Robotics and Computer-Integrated Manufacturing, 2023, 82: 102537.
[5] KUMAR N, LEE S C. Human-machine interface in smart factory: A systematic literature review[J]. Technological Forecasting and Social Change, 2022, 174: 121284.
[6] LIU H, JU Z, JI X, et al. Human motion sensing and recognition: Vol. 675[M]. Springer, 2017.
[7] ZHANG P. Human–machine interfaces[M/OL]. 2010: 527-555. DOI: 10.1016/B978-1-437 7-7807-6.10013-0.
[8] TREJOS A, PATEL R, NAISH M. Force sensing and its application in minimally invasive surgery and therapy: a survey[J]. Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science, 2010, 224(7): 1435-1454.
[9] KU S, SONG B H, PARK T, et al. Soft modularized robotic arm for safe human–robot interaction based on visual and proprioceptive feedback[J]. The International Journal of Robotics Research, 2024: 02783649241227249.
[10] VOULOUTSI V, COMINELLI L, DOGAR M, et al. Towards living machines: current and future trends of tactile sensing, grasping, and social robotics[J]. Bioinspiration & biomimetics, 2023, 18(2): 025002.
[11] DONG W, WANG Y, ZHOU Y, et al. Soft human–machine interfaces: design, sensing and stimulation[J]. International Journal of Intelligent Robotics and Applications, 2018, 2: 313- 338.
[12] DE BARRIE D, PANDYA M, PANDYA H, et al. A deep learning method for vision based force prediction of a soft fin ray gripper using simulation data[J]. Frontiers in Robotics and AI, 2021, 8: 631371.
[13] ELGENEIDY K, LIGHTBODY P, PEARSON S, et al. Characterising 3D-printed soft fin ray robotic fingers with layer jamming capability for delicate grasping[C]//2019 2nd IEEE International Conference on Soft Robotics (RoboSoft). IEEE, 2019: 143-148.
[14] REHAN M, SALEEM M M, TIWANA M I, et al. A soft multi-axis high force range magnetic tactile sensor for force feedback in robotic surgical systems[J]. Sensors, 2022, 22(9): 3500
[15] FARIS O, MUTHUSAMY R, RENDA F, et al. Proprioception and exteroception of a soft robotic finger using neuromorphic vision-based sensing[J]. Soft Robotics, 2023, 10(3): 467- 481.
[16] THURUTHEL T G, SHIH B, LASCHI C, et al. Soft robot perception using embedded soft sensors and recurrent neural networks[J]. Science Robotics, 2019, 4(26): eaav1488.
[17] 刘佳, 李娜, 郝子岩, 等. 柔性机械手设计与夹取力动态特性仿真分析[J]. 河北大学学报 (自然科学版), 2018, 38(2): 119.
[18] ZHANG Z, ZHOU J, YI B, et al. A flexible swallowing gripper for harvesting apples and its grasping force sensing model[J]. Computers and Electronics in Agriculture, 2023, 204: 107489.
[19] 金波, 林龙贤. 果蔬采摘欠驱动机械手爪设计及其力控制[J]. 机械工程学报, 2014, 50(19): 1-8.
[20] YANG Y, JIN K, ZHU H, et al. A 3D-printed fin ray effect inspired soft robotic gripper with force feedback[J]. Micromachines, 2021, 12(10): 1141.
[21] TRUBY R L, WEHNER M, GROSSKOPF A K, et al. Soft somatosensitive actuators via embedded 3D printing[J]. Advanced Materials, 2018, 30(15): 1706383.
[22] STANO G, OVY S A I, EDWARDS J R, et al. One-shot additive manufacturing of robotic finger with embedded sensing and actuation[J]. The International Journal of Advanced Manufacturing Technology, 2023, 124(1-2): 467-485.
[23] FARIS O, TAWK C, HUSSAIN I. SOPCAS finger: a three-dimensional printed soft finger with pneumatic chambers for simultaneous actuation, sensing, and controlled grasping[J]. Robotics Reports, 2024, 2(1): 32-42.
[24] MATSUNO T, WANG Z, HIRAI S. Real-time curvature estimation of printable soft gripper using electro-conductive yarn[C]//2017 IEEE International Conference on Real-time Computing and Robotics (RCAR). IEEE, 2017: 5-10.
[25] 王雅纯, 张小栋, 陆竹风, 等. 用于假手指尖的光纤光栅触觉力传感器研究[J]. 仪器仪表学 报, 2023(9): 124-130.
[26] SEN S, CHAKRABARTY S, TOSHNIWAL R, et al. Design of an intelligent voice controlled home automation system[J]. International Journal of Computer Applications, 2015, 121(15).
[27] WEI D, ZHANG R, SAADATZI M N, et al. Organic piezoresistive pressure sensitive robotic skin for physical human-robot interaction[C]//International Design Engineering Technical Conferences and Computers and Information in Engineering Conference: Vol. 83907. American Society of Mechanical Engineers, 2020: V001T01A013.
[28] CHENG N, PHUA K S, LAI H S, et al. Brain-computer interface-based soft robotic glove rehabilitation for stroke[J]. IEEE Transactions on Biomedical Engineering, 2020, 67(12): 3339- 3351.
[29] MILLAN J R, RENKENS F, MOURINO J, et al. Noninvasive brain-actuated control of a mobile robot by human EEG[J]. IEEE Transactions on biomedical Engineering, 2004, 51(6): 1026-1033.
[30] JEONG J H, SHIM K H, KIM D J, et al. Brain-controlled robotic arm system based on multidirectional CNN-BiLSTM network using EEG signals[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2020, 28(5): 1226-1238.
[31] GUO Y, ZHANG R, QIU W, et al. Underwater intention recognition using head motion and throat vibration for supernumerary robotic assistance[C/OL]//2023 IEEE 19th International Conference on Automation Science and Engineering (CASE). 2023: 1-6. DOI: 10.1109/CASE56687.2023.10260480.
[32] CALLUPE LUNA J, MARTINEZ ROCHA J, MONACELLI E, et al. WISP, wearable inertial sensor for online wheelchair propulsion detection[J]. Sensors, 2022, 22(11): 4221.
[33] LI Y. Hand gesture recognition using Kinect[C]//2012 IEEE International Conference on Computer Science and Automation Engineering. IEEE, 2012: 196-199.
[34] ADEBAYO O, ADETIBA E, AJAYI O. Hand gesture recognition-based control of motorized wheelchair using electromyography sensors and recurrent neural network[C]//IOP Conference Series: Materials Science and Engineering: Vol. 1107. IOP Publishing, 2021: 012063.
[35] XU J, HUANG Z, LIU L, et al. Eye-Gaze controlled wheelchair based on deep learning[J]. Sensors, 2023, 23(13): 6239.
[36] SHARIFI M, BEHZADIPOUR S, VOSSOUGHI G. Nonlinear model reference adaptive impedance control for human–robot interactions[J]. Control Engineering Practice, 2014, 32: 9-27.
[37] COLGATE E, BICCHI A, PESHKIN M A, et al. Safety for physical human-robot interaction [M]//Springer handbook of robotics. Springer, 2008: 1335-1348.
[38] HADDADIN S, CROFT E. Physical human–robot interaction[J]. Springer handbook of robotics, 2016: 1835-1874.
[39] HOGAN N. Impedance control: An approach to manipulation: Part II—Implementation[Z]. 1985.
[40] KAZEROONI H, HOUPT P, SHERIDAN T. Robust compliant motion for manipulators, Part II: Design method[J]. IEEE Journal on Robotics and Automation, 1986, 2(2): 93-105.
[41] SHARIFI M, ZAKERIMANESH A, MEHR J K, et al. Impedance variation and learning strategies in human–robot interaction[J]. IEEE Transactions on Cybernetics, 2021, 52(7): 6462-6475.
[42] AGHILI F. Robust impedance-matching of manipulators interacting with uncertain environments: application to task verification of the space station’s dexterous manipulator[J]. IEEE/ASME Transactions on Mechatronics, 2019, 24(4): 1565-1576.
[43] KIM D, KOH K, CHO G R, et al. A robust impedance controller design for series elastic actuators using the singular perturbation theory[J]. IEEE/ASME Transactions on Mechatronics, 2019, 25(1): 164-174.
[44] ANDERSON R J, SPONG M W. Hybrid impedance control of robotic manipulators[J]. IEEE Journal on Robotics and Automation, 1988, 4(5): 549-556.
[45] ABDOSSALAMI A, SIROUSPOUR S. Adaptive control for improved transparency in haptic simulations[J]. IEEE Transactions on Haptics, 2008, 2(1): 2-14.
[46] FLACCO F, KRÖGER T, DE LUCA A, et al. A depth space approach to human-robot collision avoidance[C]//2012 IEEE international conference on robotics and automation. IEEE, 2012: 338-345.
[47] PHAN S, QUEK Z F, SHAH P, et al. Capacitive skin sensors for robot impact monitoring [C]//2011 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2011: 2992-2997.
[48] AVANZINI G B, CERIANI N M, ZANCHETTIN A M, et al. Safety control of industrial robots based on a distributed distance sensor[J]. IEEE Transactions on Control Systems Technology, 2014, 22(6): 2127-2140.
[49] HAO Y, ZHANG S, FANG B, et al. A review of smart materials for the boost of soft actuators, soft sensors, and robotics applications[J]. Chinese Journal of Mechanical Engineering, 2022, 35(1): 1-16.
[50] EMERSON J, ELGENEIDY K. Optimising soft fin ray robotic fingers using finite element analysis to reduce object slippage[C]//Proceedings of The 3rd UK-RAS Conference “Robots into the real world. 2020: 43-45.
[51] CAI J, LEI T. An autonomous positioning method of tube-to-tubesheet welding robot based on coordinate transformation and template matching[J]. IEEE Robotics and Automation Letters, 2021, 6(2): 787-794.
[52] ANTMAN S S, OSBORN J E. The principle of virtual work and integral laws of motion[J]. Archive for Rational Mechanics and Analysis, 1979, 69: 231-262.
[53] KALAITZAKIS M, CAIN B, CARROLL S, et al. Fiducial markers for pose estimation: Overview, applications and experimental comparison of the artag, apriltag, aruco and stag markers[J]. Journal of Intelligent & Robotic Systems, 2021, 101: 1-26.
[54] ZHANG Z, LIU G, LI Z, et al. Flexible tactile sensors with biomimetic microstructures: Mechanisms, fabrication, and applications[J]. Advances in Colloid and Interface Science, 2023: 102988.
[55] SPENCER JR B, DYKE S, SAIN M, et al. Phenomenological model for magnetorheological dampers[J]. Journal of engineering mechanics, 1997, 123(3): 230-238.
[56] DOMINGUEZ A, SEDAGHATI R, STIHARU I. Modelling the hysteresis phenomenon of magnetorheological dampers[J]. Smart Materials and Structures, 2004, 13(6): 1351.
[57] MATHIS A T, BALAJI N N, KUETHER R J, et al. A review of damping models for structures with mechanical joints[J]. Applied Mechanics Reviews, 2020, 72(4): 040802.
[58] SHITIKOVA M V, KRUSSER A I. Models of viscoelastic materials: A review on historical development and formulation[J]. Theoretical Analyses, Computations, and Experiments of Multiscale Materials: A Tribute to Francesco Dell’Isola, 2022: 285-326.
[59] DENAVIT J, HARTENBERG R S. A kinematic notation for lower-pair mechanisms based on matrices[M]. American Society of Mechanical Engineers, 1955.
[60] 李宪华, 孙青, 张雷刚, 等. 基于 SVD 可操作度指标的机械臂灵活性分析[J]. 制造技术与 机床, 2019(4): 29-34.
[61] SONG P, YU Y, ZHANG X. Impedance control of robots: an overview[C]//2017 2nd international conference on cybernetics, robotics and control (CRC). IEEE, 2017: 51-55.
[62] CHU T, NGUYEN T, YOO H, et al. A review of vibration analysis and its applications[J]. Heliyon, 2024.
[63] DIMEAS F, ASPRAGATHOS N. Online stability in human-robot cooperation with admittance control[J]. IEEE transactions on haptics, 2016, 9(2): 267-278.