多目结构光高精度3D机器视觉传感技术研究及应用

随着工业生产朝着智能制造方向发展，工业产品的质量检测随之成为行业关注和发展的重点内容。工业生产中的质量检测主要包含几何尺寸测量、表面缺陷检测等内容。而主要的测量方案为基于2D平面测量和单目结构光3D 重建测量方案。常规的2D 测量方案在高度、角度及平面度等测量尺度上不能发挥很好地效果，3D测量方案应运而生。其中主流的测量方案为基于面结构光的单目高精度3D测 量。通过结构光投影模组投射条纹光栅图片，并由相机进行采集，可以得到对应 的被测物体在空间中的三维模型。单目结构光相机在测量过程中易受到遮挡且视野由单个相机视野决定，视野的遮挡和数据的缺失导致测量准确性下降和效率的降低。

本文在综合分析结构光相机的成像因素后，针对遮挡、视野缺失等问题以及测量速率进行分析，选择多相机单投影的技术路线，对多目结构光系统的硬件、标定及重建应用进行分析总结，搭建并设计了一套基于多目结构光系统的高精度测量方案。在硬件搭建方面，本文根据实际应用场景与需求，对被测物体的多个角度进行测量。同时，本文针对结构光系统的关键技术，对结构光编码方案和系统标定方法进行研究和实验。在分析各类结构光编码方案的原理与特点后，针对高精 度的测量需求优化时间编码结构光图案和序列的设计。在结构光系统标定方法中，本文系统性地优化了结构光相机标定方法与流程，使标定过程中对相机和投影模 组的位置没有严格的要求，同时结合编码结构光对重建精度的影响因素，提高了标定过程的精度和算法稳定性。本文根据实际工业生产环境中的测量条件，设计完成了针对工业产线的实际测量功能和应用。经过实验测试和论证，本文所设计的多目结构光系统可以实现亚微米级的测量精度，并且可以完成大视野和多视角的测量效果，提高了实际工业生产条件下的多目结构光测量效率。

With the development of industrial production towards intelligent manufacturing, quality inspection of industrial products has become a key content of industry attention and development. Quality inspection in industrial production mainly includes geometric dimension measurement, surface defect detection and other contents. And the main measurement scheme is based on 2D plane measurement and monocular structured light 3D reconstruction measurement scheme. The conventional 2D measurement scheme does not perform well in height, angle and flatness measurement scales, and 3D measurement scheme emerges. Among them, the mainstream measurement scheme is based on surface structured light monocular high-precision 3D measurement. By projecting stripe grating images with structured light projection module and grabbing them by camera, the corresponding three-dimensional model of the measured object in space can be obtained. Monocular structured light camera is easy to be affected by occlusion and the field of view is determined by a single camera in the measurement process. The occlusion of the field of view and the lack of data lead to the decrease of measurement accuracy and efficiency.

This paper comprehensively analyzes the imaging factors of structured light camera, such as camera view, occlusion. Then the paper chooses the technical route of multi-occular single projection, analyzes and summarizes the hardware, calibration and reconstruction application of multi-occular structured light system. The paper designed a system of high-precision measurement scheme based on multi-occular structured light system. In terms of hardware construction, this paper measures the measured object from multiple angles according to the actual application scenario and demand. At the same time, this paper studies and experiments on the key technologies of structured light system, such as structured light encoding scheme and calibration method. After analyzing the principles and characteristics of various structured light coding schemes, this paper optimizes the design of time-coded structured light pattern and sequence for high-precision measurement requirements. In the method of structured light system calibration, this paper systematically optimizes the method and process of structured light camera calibration. There is no strict requirement for the position of camera and projection module in the calibration process. At the same time, combined with the influencing factors of coded structured light on reconstruction accuracy, this paper improves the accuracy and algorithm stability of calibration process. According to the actual measurement conditions in industrial production environment, this paper designs and completes the actual measurement function and application for industrial production line. After experimental testing and demonstration, this paper designs a multi-occular structured light system that can achieve sub-micron level measurement accuracy, and can complete large field of view and multi-angle measurement effect, improving the efficiency of multi-occular structured light measurement under actual industrial production conditions.

[1] ZHONG K, LI Z, ZHOU X, et al. Enhanced phase measurement profilometry for industrial 3D inspection automation[J]. The International Journal of Advanced Manufacturing Technology, 2015, 76: 1563-1574.
[2] 于晓洋, 张键, 吴丽莹, 等. 结构光三维视觉技术的进展[J]. 宇航计测技术, 1997(43-48).
[3] 党成亮. 基于结构光和自编码学习的工件三维重建[D]. 武汉科技大学, 2021.
[4] 曲学军, 张璐. 基于双目视觉的三维测量方法[J]. 计算机仿真, 2011(373-377).
[5] 张静, 吕柳, 闫爱民. 结构光三维成像技术研究进展[J]. 上海师范大学学报 (自然科学版), 2021, 50(4): 391-401.
[6] 陈红. 基于双目立体视觉几何量的高精度测量技术研究[D]. 贵州民族大学, 2022.
[7] 邵双运. 光学三维测量技术与应用[J]. 现代仪器, 2008, No.76(10-13).
[8] EL-HAKIM S F, BERALDIN J A, BLAIS F. Comparative evaluation of the performance of passive and active 3D vision systems[C]//Digital Photogrammetry and Remote Sensing’95: volume 2646. SPIE, 1995: 14-25.
[9] MOTTA J M S, DE CARVALHO G C, MCMASTER R. Robot calibration using a 3D visionbased measurement system with a single camera[J]. Robotics and Computer-Integrated Manufacturing, 2001, 17(6): 487-497.
[10] 吴梦娟. 多目视觉三维测量方法研究[Z]. 哈尔滨工业大学, 2019.
[11] LEE H M, CHOI W C. Algorithm of 3D spatial coordinates measurement using a camera image[J]. Journal of Image and Graphics, 2015, 3(1): 30-33.
[12] 樊强, 陈大为, 习俊通. 高精度激光点扫描三维测量系统及应用[J]. 上海交通大学学报, 2006(227-230).
[13] 卫晓鑫. 基于线结构光的多视角三维重建与拼接方法研究[Z]. 天津工业大学, 2021.
[14] 邓悦星. 基于激光测量技术和视觉测量技术的汽车零部件在线测量系统分析[J]. 现代制造技术与装备, 2023, 59(189-191).
[15] 赵楠. 面结构光三维测量系统关键技术研究[J]. 科技传播, 2021, 13(11): 157-159.
[16] 熊宗刚. 结构光双目三维成像关键技术研究[Z]. 电子科技大学, 2021.
[17] 李栋. 结构光视觉高精度测量与工件位姿识别[Z]. 浙江大学, 2020.
[18] ZHANG L, CURLESS B, SEITZ S. Rapid shape acquisition using color structured light and multi-pass dynamic programming[C]//Proceedings. First International Symposium on 3D Data Processing Visualization and Transmission. 2002: 24-36.
[19] FERNANDEZ S, SALVI J, PRIBANIC T. Absolute phase mapping for one-shot dense pattern projection[C]//2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops. IEEE, 2010: 64-71.
[20] WONG A, NIU P, XIANG H. Fast acquisition of dense depth data by a new structured light scheme[J]. Computer Vision & Image Understanding, 2005, 98(3): 398-422.
[21] LIU X, KOFMAN J. Background and amplitude encoded fringe patterns for 3D surface-shape measurement[J]. Optics and Lasers in Engineering, 2017, 94: 63-69.
[22] YIN W, FENG S, TAO T, et al. Calibration method for panoramic 3D shape measurement with plane mirrors[Z]. 13.
[23] Absolute phase recovery in structured light illumination systems: Sinusoidal vs. intensity discrete patterns[J]. Optics and Lasers in Engineering, 2016, 84: 111-119.
[24] LIU D, PAN Y, LU R. FPGA-assisted high-precision, high-speed 3D shape measurement[J]. Sensors and Actuators A: Physical, 2020, 315: 112366.
[25] YIN W, FENG S, TAO T, et al. High-speed 3D shape measurement using the optimized composite fringe patterns and stereo-assisted structured light system[J]. Opt. Express, 2019, 27(3): 2411-2431.
[26] DIPANDA A, WOO S. Towards a real-time 3D shape reconstruction using a structured light system[J]. Pattern Recognition, 2005, 38(10): 1632-1650.
[27] ZUO C, TAO T, FENG S, et al. Micro Fourier Transform Profilometry (μFTP): 3D shape measurement at 10,000 frames per second[J]. Optics and Lasers in Engineering, 2018, 102: 70-91.
[28] HU H, GAO J, ZHOU H, et al. A combined binary defocusing technique with multi-frequency phase error compensation in 3D shape measurement[J]. Optics and Lasers in Engineering, 2020, 124: 105806.
[29] LIU X, SONG L, ZHANG M, et al. 3D structured light scanning with phase domain-modulated fringe patterns[J/OL]. IEEE Transactions on Industrial Electronics, 2023, 70(5): 5245-5254. DOI: 10.1109/TIE.2022.3187597.
[30] NGUYEN H, LY K L, TRAN T, et al. hNet: Single-shot 3D shape reconstruction using structured light and h-shaped global guidance network[J]. Results in Optics, 2021, 4: 100104.
[31] LYU N, YU H, HAN J, et al. Structured light-based underwater 3D reconstruction techniques: A comparative study[J]. Optics and Lasers in Engineering, 2023, 161: 107344.
[32] SARAFRAZ A, HAUS B K. A structured light method for underwater surface reconstruction [J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2016, 114: 40-52.
[33] MASSOT-CAMPOS M, OLIVER-CODINA G. Underwater laser-based structured light system for one-shot 3D reconstruction[C]//SENSORS, 2014 IEEE. 2014: 1138-1141.
[34] MAKHSOUS S, MOHAMMAD H M, SCHENK J M, et al. A novel mobile structured light system in food 3D reconstruction and volume estimation[J]. Sensors, 2019, 19(3).
[35] LEROUX T, IENG S H, BENOSMAN R. Event-based structured light for depth reconstruction using frequency tagged light patterns[A]. 2018. arXiv: 1811.10771.
[36] ROCCHINI C, CIGNONI P, MONTANI C, et al. A low cost 3D scanner based on structured light[C]//computer graphics forum: volume 20. Wiley Online Library, 2001: 299-308.
[37] KUNZMANN H, TRAPET E, WÄLDELE F. A uniform concept for calibration, acceptance test, and periodic inspection of coordinate measuring machines using reference objects[J]. CIRP annals, 1990, 39(1): 561-564.
[38] ZHANG S. High-speed 3D shape measurement with structured light methods: A review[J]. Optics and Lasers in Engineering, 2018, 106: 119-131.
[39] TAO T, CHEN Q, FENG S, et al. High-precision real-time 3D shape measurement using a bi-frequency scheme and multi-view system[J]. Applied Optics, 2017, 56(13): 3646.
[40] GENG J. Structured-light 3D surface imaging: a tutorial[J]. Advances in Optics and Photonics, 2011, 3(2): 128-160.
[41] MURPHY Q. Accelerating structured light surface reconstruction with motion analysis[C]// 2018.
[42] ALTALIB A. Depth map extraction using structured light[D]. 2019.
[43] 陆军, 张鑫, 高乐, 等. 基于 De Bruijn 序列的彩色结构光编解码方法研究[J]. 光电子. 激光, 2014, 25(1): 147-155.
[44] KAWASAKI H, FURUKAWA R, SAGAWA R, et al. Dynamic scene shape reconstruction using a single structured light pattern[C]//2008 IEEE conference on computer vision and pattern recognition. Ieee, 2008: 1-8.
[45] PAGES J, SALVI J, FOREST J. A new optimised De Bruijn coding strategy for structured light patterns[C]//Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004.: volume 4. IEEE, 2004: 284-287.
[46] 何春桥. 基于多频外差法的双目结构光三维重建方法研究[Z]. 重庆大学, 2018.
[47] 章登极. 基于面结构光立体视觉的三维测量技术研究[Z]. 深圳大学, 2018.
[48] VILACA J L, FONSECA J C, PINHO A M. Calibration procedure for 3D measurement systems using two cameras and a laser line[J]. Optics & Laser Technology, 2009, 41(2): 112-119.
[49] BROWN D C. Close-range camera calibration[J]. PE, 1971, 37.
[50] ZHANG Z. A flexible new technique for camera calibration[J]. IEEE Transactions on pattern analysis and machine intelligence, 2000, 22(11): 1330-1334.
[51] HUANG P S. Novel method for structured light system calibration[J]. Optical Engineering, 2006, 45(8): 083601.
[52] MORENO D, TAUBIN G. Simple, accurate, and robust projector-camera calibration[C]//2012 Second International Conference on 3D Imaging, Modeling, Processing, Visualization & Transmission. 2012: 464-471.
[53] 盖绍彦, 达飞鹏. 一种新的相位法三维轮廓测量系统模型及其标定方法研究[J]. 自动化学报, 2007(9): 902-910.
[54] 杨海清, 王洋洋. 基于面结构光几何关系的三维数字化研究[J]. 计算机应用研究, 2018, 35(7): 2237-2240.
[55] 何剑汇. 基于双目结构光的工件点云构建与多视角配准方法研究[Z]. 山东大学, 2021.
[56] HOCKEN R J, PEREIRA P H, et al. Coordinate measuring machines and systems: volume 2[M]. CRC press Boca Raton, 2012.
[57] FURUNO T, FUJITA S, DONGHAO W, et al. 3D bow and posture measurements for virtual reality customer service training system[J]. Journal of Image and Graphics, 2021, 9(4): 152-156.
[58] GALLUCCI A, ZNAMENSKIY D, PETKOVIC M. Prediction of 3D body parts from face shape and anthropometric measurements[J]. Journal of Image and Graphics, 2020, 8(3): 67-74.
[59] AILANI V, PRAKASH D, VENKATESH K. Self localization with edge detection in 3D space [J]. Journal of Image and Graphics, 2013, 1(2): 99-103.