基于双目视觉的机翼形变测量技术研究

机翼的形变检测在飞行器气动特性探究和飞行器安全保障工作中具有重要意义。基于视觉的测量技术因其具有非接触、时效性好和高精度等优势，可以实时反演物体的三维形变，近年来得到了快速发展与广泛应用。为了验证视觉测量技术应用于机翼形变检测的可行性，本文在充分调研的基础上，搭建了双目视觉形变测量系统，对立体匹配与点云重建和多视角点云配准技术进行深入研究。
对于双目图像的立体匹配技术，本文探究了传统匹配算法和神经网络两种求解视差关系的方法。首先对传统的基于区域匹配的BM和SGBM算法进行推导，然后在真实场景中应用并分析了其存在较大误差的原因。为提高立体匹配精度，本文设计了一个用于双目立体匹配网络，该网络采用了残差结构用于丰富局部特征信息，同时加入了Transformer模块来增强特征的上下文信息。通过在公开数据集上对网络模型进行训练，并在真实双目图像中进行验证。通过对比发现，该网络求解的视差在精度上远远高度传统算法，结合相机标定能够很好地恢复重建点云。
对于多视角点云配准技术，本文探究了传统手工特征匹配和神经网络高维特征匹配两种方法。本文首先对点云传统特征描述符FPFH进行推导，结合经典的ICP配准算法对多视角下的点云进行配准，然后分析该算法在部分场景表现不佳的原因。为适应机翼低纹理、多重复区域的特点，本文设计了一个用于多视角点云配准的网络，该网络采用Encode-Decode架构，通过在网络中加入跳跃连接和反卷积层实现点云局部和整体特征的融合，利用稀疏卷积降低网络的计算复杂度。通过在公开数据集上对网络进行训练，并在真实多视角点云中进行验证。结果表明本文设计的网络在多视角点云配准上取得了比传统方法更好的效果。
最后，本文对所设计的系统在实际机翼的形变检测和点云配准的应用进行了验证。通过在机翼表面设立多个测量点，在末端施加载荷模拟形变，将机翼形变前后的离面位移与真实位移进行对比。结果表明本文提出的形变检测系统取得了良好的精度。同时，针对景深和视野因素带来的远场精度不佳问题，本文在不同角度下拍摄获取了机翼的局部点云，通过本文探究的点云配准方法实现多视角点云配准，然后对比验证配准前后远场的测量误差，结果证明本文提出的配准网络能够实现机翼的全场模型重建并提高形变测量精度。

[1] 包为民. 航天飞行器控制技术研究现状与发展趋势[J]. 自动化学报, 2013, 39(6): 697-702.
[2] 陈宗基, 张汝麟, 张平, 等. 飞行器控制面临的机遇与挑战[J]. 自动化学报, 2013, 39(6): 703-710.
[3] MOLENT L, DIXON B. Airframe metal fatigue revisited[J]. International Journal of Fatigue, 2020, 131: 105323.
[4] CHULIANG Y, KEGE L. Theory of economic life prediction and reliability assessment of aircraft structures[J]. Chinese Journal of Aeronautics, 2011, 24(2): 164-170.
[5] GONG X, BANSMER S. Horn–Schunck optical flow applied to deformation measurement of a birdlike airfoil[J]. Chinese Journal of Aeronautics, 2015, 28(5): 1305-1315.
[6] BLACK J T, PITCHER N A, REEDER M F, et al. Videogrammetry dynamics measurements of a lightweight flexible wing in a wind tunnel[J]. Journal of aircraft, 2010, 47(1): 172-180.
[7] BURNER A, LIU T, DELOACH R. Uncertainty of videogrammetric techniques used for aerodynamic testing[C]//22nd AIAA aerodynamic measurement technology and ground testing conference. 2002: 2794.
[8] 张华强, 刘林, 秦昌礼, 等. 基于分布式惯导测量的机翼弹性形变算法[J]. 北京邮电大学学报, 44(6): 33.
[9] WU X, XU Z, ZENG J. Reconstruction of Wing Structure Deformation Based on Particle Swarm Optimization Ridge Regression[J]. International Journal of Aerospace Engineering, 2022, 2022.
[10] KO W L, RICHARDS W L, TRAN V T. Displacement theories for in-flight deformed shape predictions of aerospace structures[R]. 2007.
[11] MA Z, CHEN X. Fiber Bragg gratings sensors for aircraft wing shape measurement: Recent applications and technical analysis[J]. Sensors, 2018, 19(1): 55.
[12] WISSMAN J, PEREZ-ROSADO A, EDGERTON A, et al. New compliant strain gauges for selfsensing dynamic deformation of flapping wings on miniature air vehicle[J]. Smart materials and structures, 2013, 22(8): 085031.
[13] BODEN F, BODENSIEK K, STASICKI B. Application of image pattern correlation for nonintrusive deformation measurements of fast rotating objects on aircrafts[C]//Fourth International Conference on Experimental Mechanics: volume 7522. SPIE, 2010: 789-798.
[14] 马振. 基于 FBG 的机翼分布式测量技术研究[D]. 中南大学, 2017.
[15] ZHU Z, ZHANG M, ZHOU X. A new baseline measurement method for multinode and multibaseline interferometric SAR systems using fiber Bragg gratings[J]. IEEE Transactions on Aerospace and Electronic Systems, 2019, 58(1): 4-16.
[16] SORIANO-AMAT M, FRAGAS-SÁNCHEZ D, MARTINS H F, et al. Monitoring of a highly flexible aircraft model wing using time-expanded phase-sensitive OTDR[J]. Sensors, 2021, 21(11): 3766.
[17] 李小路, 江月松, 何云涛. 机翼变形的双激光器实时测量原理[J]. 光学技术, 2006, 32(3):333-336.
[18] LI X, XU L, TAN C, et al. Real-time measurement of aerodynamic deformation of wing by laser rangefinder[C]//2010 IEEE Instrumentation & Measurement Technology Conference Proceedings. IEEE, 2010: 1581-1584.
[19] 班建华, 汪西, 刘思仁, 等. 组合式测量技术在飞机部件形位检测中的应用[J]. 现代制造工程, 2019(2): 100-104.
[20] ALDAO E, GONZÁLEZ-JORGE H, PÉREZ J. Metrological comparison of LiDAR and photogrammetric systems for deformation monitoring of aerospace parts[J]. Measurement, 2021, 174: 109037.
[21] 季红侠. 飞机大部件对接中的自动测量技术研究与系统开发 [D][D]. 南京: 南京航空航天大学, 2012.
[22] PILCH A, MAHAJAN A, CHU T. Measurement of whole-field surface displacements and strain using a genetic algorithm based intelligent image correlation method[J]. J. Dyn. Sys., Meas., Control, 2004, 126(3): 479-488.
[23] MOLINA-VIEDMA A J, LÓPEZ-ALBA E, FELIPE-SESÉ L, et al. Operational deflection shape extraction from broadband events of an aircraft component using 3D-DIC in magnified images[J]. Shock and Vibration, 2019, 2019.
[24] SOUSA P J, BARROS F, TAVARES P J, et al. Displacement measurement and shape acquisition of an RC helicopter blade using Digital Image Correlation[J]. Procedia Structural Integrity, 2017, 5: 1253-1259.
[25] WU J, YU Z, WANG T, et al. Airplane wing deformation and flight flutter detection method by using three-dimensional speckle image correlation technology[J]. JOSA A, 2017, 34(6): 924-930.
[26] STASICKI B, BODEN F. In-flight measurements of aircraft propeller deformation by means of an autarkic fast rotating imaging system[C]//International Conference on Experimental Mechanics 2014: volume 9302. SPIE, 2015: 412-417.
[27] LI L G, LIANG J, GUO X, et al. Full-field wing deformation measurement scheme for inflight cantilever monoplane based on 3D digital image correlation[J]. Measurement Science and Technology, 2014, 25(6): 065202.
[28] WANG L, BI S, LU X, et al. Deformation measurement of high-speed rotating drone blades based on digital image correlation combined with ring projection transform and orientation codes[J]. Measurement, 2019, 148: 106899.
[29] FAYYAD T M, LEES J M. Application of digital image correlation to reinforced concrete fracture[J]. Procedia Materials Science, 2014, 3: 1585-1590.
[30] LIU Y, SU X, GUO X, et al. A novel concentric circular coded target, and its positioning and identifying method for vision measurement under challenging conditions[J]. Sensors, 2021, 21 (3): 855.
[31] SOUTO JANEIRO A, FERNÁNDEZ LÓPEZ A, CHIMENO MANGUAN M, et al. ThreeDimensional Digital Image Correlation Based on Speckle Pattern Projection for Non-Invasive Vibrational Analysis[J]. Sensors, 2022, 22(24): 9766.
[32] MARR D. A computational investigation into the human representation and processing of visual information[J]. Vision, 1982: 125-126.
[33] WALLACH H, O’CONNELL D. The kinetic depth effect.[J]. Journal of experimental psychology, 1953, 45(4): 205.
[34] 丁少闻, 张小虎, 于起峰, 等. 非接触式三维重建测量方法综述[J]. 激光与光电子学进展, 2017, 54(7): 070003.
[35] 梁磊. 基于双目视觉的小型飞机位姿估计[Z]. 中国民用航空飞行学院, 2021.
[36] LIU Y, GE Z, YUAN Y, et al. Wing deformation measurement using the stereo-vision methods in the presence of camera movements[J]. Aerospace Science and Technology, 2021, 119: 107161.
[37] 赵浚壹. 基于深度相机的无人机三维场景重建技术研究[D]. 桂林电子科技大学, 2021.
[38] ZOLLHÖFER M, NIESSNER M, IZADI S, et al. Real-time non-rigid reconstruction using an RGB-D camera[J]. ACM Transactions on Graphics (ToG), 2014, 33(4): 1-12.
[39] NESBIT P R, HUGENHOLTZ C H. Enhancing UAV–SFM 3D model accuracy in high-relief landscapes by incorporating oblique images[J]. Remote Sensing, 2019, 11(3): 239.
[40] SHANG H, LIU C, WANG R. Measurement methods of 3D shape of large-scale complex surfaces based on computer vision: A review[J]. Measurement, 2022: 111302.
[41] SEITZ S M, CURLESS B, DIEBEL J, et al. A comparison and evaluation of multi-view stereo reconstruction algorithms[C]//2006 IEEE computer society conference on computer vision and pattern recognition (CVPR’06): volume 1. IEEE, 2006: 519-528.
[42] 隋婧, 金伟其. 双目立体视觉技术的实现及其进展[J]. 电子技术应用, 2004, 30(10): 4-6.
[43] 赵晨园, 李文新, 张庆熙. 双目视觉的立体匹配算法研究进展[J]. 计算机科学与探索, 2020,14(7): 1104-1113.
[44] DO P N B, NGUYEN Q C. A review of stereo-photogrammetry method for 3-D reconstruction in computer vision[C]//2019 19th International Symposium on Communications and Information Technologies (ISCIT). IEEE, 2019: 138-143.
[45] MATTOCCIA S. Stereo vision: Algorithms and applications[J]. University of Bologna, 2011, 22.
[46] HIRSCHMULLER H, SCHARSTEIN D. Evaluation of stereo matching costs on images with radiometric differences[J]. IEEE transactions on pattern analysis and machine intelligence, 2008, 31(9): 1582-1599.
[47] LOWE D G. Distinctive image features from scale-invariant keypoints[J]. International journal of computer vision, 2004, 60: 91-110.
[48] BAY H, TUYTELAARS T, VAN GOOL L. Surf: Speeded up robust features[J]. Lecture notes in computer science, 2006, 3951: 404-417.
[49] RUBLEE E, RABAUD V, KONOLIGE K, et al. ORB: An efficient alternative to SIFT or SURF [C]//2011 International conference on computer vision. Ieee, 2011: 2564-2571.
[50] HIRSCHMULLER H. Stereo processing by semiglobal matching and mutual information[J]. IEEE Transactions on pattern analysis and machine intelligence, 2007, 30(2): 328-341.
[51] YAO G, CUI J, DENG K, et al. Robust Harris corner matching based on the quasi-homography transform and self-adaptive window for wide-baseline stereo images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 56(1): 559-574.
[52] SAN T T, WAR N. Stereo matching algorithm by hill-climbing segmentation[C]//2017 IEEE 6th Global Conference on Consumer Electronics (GCCE). IEEE, 2017: 1-2.
[53] CHU T, RANSON W, SUTTON M A. Applications of digital-image-correlation techniques to experimental mechanics[J]. Experimental mechanics, 1985, 25: 232-244.
[54] SUN D, YANG X, LIU M Y, et al. Pwc-net: Cnns for optical flow using pyramid, warping, and cost volume[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 8934-8943.
[55] GUO X, YANG K, YANG W, et al. Group-wise correlation stereo network[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019: 3273-3282.
[56] ZBONTAR J, LECUN Y, et al. Stereo matching by training a convolutional neural network to compare image patches.[J]. J. Mach. Learn. Res., 2016, 17(1): 2287-2318.
[57] CHEN L, FAN L, CHEN J, et al. A full density stereo matching system based on the combination of CNNs and slanted-planes[J]. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2017, 50(2): 397-408.
[58] LUO W, SCHWING A G, URTASUN R. Efficient deep learning for stereo matching[C]// Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 5695-5703.
[59] MAYER N, ILG E, HAUSSER P, et al. A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 4040-4048.
[60] KENDALL A, MARTIROSYAN H, DASGUPTA S, et al. End-to-end learning of geometry and context for deep stereo regression[C]//Proceedings of the IEEE international conference on computer vision. 2017: 66-75.
[61] ZHANG F, PRISACARIU V, YANG R, et al. Ga-net: Guided aggregation net for end-to-end stereo matching[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019: 185-194.
[62] HUANG X, MEI G, ZHANG J, et al. A comprehensive survey on point cloud registration[A]. 2021.
[63] AIGER D, MITRA N J, COHEN-OR D. 4-points congruent sets for robust pairwise surface registration[M]//ACM SIGGRAPH 2008 papers. 2008: 1-10.
[64] RUSU R B, BLODOW N, MARTON Z C, et al. Aligning point cloud views using persistent feature histograms[C]//2008 IEEE/RSJ international conference on intelligent robots and systems. IEEE, 2008: 3384-3391.
[65] 顾旭波, 张永举, 张健, 等. 基于 SIFT 算法及阈值筛选的点云配准技术研究[J]. 计算机测量与控制, 2017, 25(12): 247-250.
[66] MELLADO N, AIGER D, MITRA N J. Super 4pcs fast global pointcloud registration via smart indexing[C]//Computer graphics forum: volume 33. Wiley Online Library, 2014: 205-215.
[67] CHEN Y, MEDIONI G. Object modelling by registration of multiple range images[J]. Image and vision computing, 1992, 10(3): 145-155.
[68] MARTULL S, PERIS M, FUKUI K. Realistic CG stereo image dataset with ground truth disparity maps[C]//ICPR workshop TrakMark2012: volume 111. 2012: 117-118.
[69] RUSINKIEWICZ S, LEVOY M. Efficient variants of the ICP algorithm[C]//Proceedings third international conference on 3-D digital imaging and modeling. IEEE, 2001: 145-152.
[70] KHOSHELHAM K. Closed-form solutions for estimating a rigid motion from plane correspondences extracted from point clouds[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2016, 114: 78-91.
[71] ZHU H, GUO B, ZOU K, et al. A review of point set registration: From pairwise registration to groupwise registration[J]. Sensors, 2019, 19(5): 1191.
[72] HE Y, YANG J, LI Z, et al. ICP registration based on 3D point clouds feature descriptor[C]// Twelfth International Conference on Graphics and Image Processing (ICGIP 2020): volume 11720. SPIE, 2021: 196-205.
[73] LU W, WAN G, ZHOU Y, et al. Deepvcp: An end-to-end deep neural network for point cloud registration[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019: 12-21.
[74] QI C R, YI L, SU H, et al. Pointnet++: Deep hierarchical feature learning on point sets in a metric space[J]. Advances in neural information processing systems, 2017, 30.
[75] GOJCIC Z, ZHOU C, WEGNER J D, et al. Learning multiview 3d point cloud registration[C]// Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020: 1759-1769.
[76] KAEHLER A, BRADSKI G. Learning OpenCV 3: computer vision in C++ with the OpenCV library[M]. ” O’Reilly Media, Inc.”, 2016.
[77] HARTLEY R I. Theory and practice of projective rectification[J]. International Journal of Computer Vision, 1999, 35(2): 115.
[78] BOUGUET J Y, PERONA P. Camera calibration from points and lines in dual-space geometry [C]//Proc. 5th European Conf. on Computer Vision. Citeseer, 1998: 2-6.
[79] SCHARSTEIN D, SZELISKI R. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms[J]. International journal of computer vision, 2002, 47: 7-42.
[80] 一颗小树 x. 立体匹配算法[EB/OL]. 2021. https://bbs.huaweicloud.com/blogs/313208.
[81] 吕明哲. 什么是体素 (Voxel)?[EB/OL]. 2021. https://zhuanlan.zhihu.com/p/348563616.
[82] SI H, QIU J, LI Y. A review of point cloud registration algorithms for laser scanners: applications in large-scale aircraft measurement[J]. Applied Sciences, 2022, 12(20): 10247.
[83] RUSU R B, BLODOW N, BEETZ M. Fast point feature histograms (FPFH) for 3D registration [C]//2009 IEEE international conference on robotics and automation. IEEE, 2009: 3212-3217.
[84] BELLO S A, YU S, WANG C, et al. Deep learning on 3D point clouds[J]. Remote Sensing, 2020, 12(11): 1729.
[85] QI C R, SU H, MO K, et al. Pointnet: Deep learning on point sets for 3d classification and segmentation[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 652-660.
[86] DENG H, BIRDAL T, ILIC S. Ppfnet: Global context aware local features for robust 3d point matching[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 195-205.
[87] AOKI Y, GOFORTH H, SRIVATSAN R A, et al. Pointnetlk: Robust & efficient point cloud registration using pointnet[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019: 7163-7172.
[88] LUCAS B D. Generalized image matching by the method of differences[M]. Carnegie Mellon University, 1985.
[89] WANG Y, SOLOMON J M. Deep closest point: Learning representations for point cloud registration[C]//Proceedings of the IEEE/CVF international conference on computer vision. 2019: 3523-3532.
[90] DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16x16 words:Transformers for image recognition at scale[A]. 2020.
[91] HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 770-778.
[92] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[J]. Advances in neural information processing systems, 2017, 30.
[93] SUN J, SHEN Z, WANG Y, et al. LoFTR: Detector-free local feature matching with transformers[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2021: 8922-8931.
[94] Forrest Ding. 【点云局部特征描述子】PFH & FPFH)?[EB/OL]. 2020. https://https://zhuanl an.zhihu.com/p/192343758.
[95] RONNEBERGER O, FISCHER P, BROX T. U-net: Convolutional networks for biomedical image segmentation[C]//Medical Image Computing and Computer-Assisted Intervention– MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18. Springer, 2015: 234-241.
[96] LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2015: 3431-3440.
[97] GRAHAM B, ENGELCKE M, VAN DER MAATEN L. 3d semantic segmentation with submanifold sparse convolutional networks[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 9224-9232.
[98] CHOY C, PARK J, KOLTUN V. Fully convolutional geometric features[C]//Proceedings of the IEEE/CVF international conference on computer vision. 2019: 8958-8966.