面向集装箱港口自动装卸场景的视觉定位系统研究

为了提高港口的装卸效率和安全性，降低作业成本并提高港口的吞吐量和贸易流通效率，需要提升集装箱港口的装卸自动化程度，故集装箱港口需要引入依托于感知、视觉、通讯等技术的定位系统帮助港口进行集装箱和起升机构吊具的位置确定。本课题为此场景设计了一套分层级的视觉定位方案，包括粗定位视觉定位系统、精定位视觉定位系统和补盲系统，以不同的视野和定位精度协同实现此场景下的集装箱及起升机构吊具的定位。本课题的研究内容如下：

针对港口工况场景进行了材料分析和选择。选择高灵敏度、高动态范围、低噪声、高帧率、高分辨率、宽温度范围的CMOS图像传感器材料；选择具有抗反射、增透的光学镜头镀膜材料，以适应集装箱港口的多种场景且稳定成像；分析并选择系统外壳及连接结构的材料，以保证视觉系统在恶劣环境下长期稳定工作。

从粗定位视觉定位系统、精定位视觉定位系统和补盲系统三部分的定位视野和定位精度出发，针对这三部分子系统分别进行硬件选型和系统设计，对多个视觉系统进行相机参数标定和系统参数标定，建立起以港口起重机某固定点为基准的空间坐标系，为后续的定位算法设计与实现提供硬件验证平台。

为了确定集装箱精确位置，粗定位视觉定位系统对集装箱进行全局视野下的定位，获取集装箱的堆区轮廓，在此基础上，精定位视觉定位系统对目标集装箱进行局部视野下的进一步精准定位，获取集装箱更精准的空间位置信息。

为了实时检测定位起升机构吊具，本课题设计的视觉定位系统中由精定位视觉定位系统和补盲系统分别对起升机构吊具进行定位，且引入ArUco标记物作为人工标记物简化吊具定位任务。精定位视觉定位系统负责在吊具正常下放抓箱过程中对吊具进行实时动态检测跟踪；在吊具收起或未完全下放时，即吊具处于精定位视觉定位系统盲区内时，由补盲系统对起升机构吊具位置进行补充定位。

本课题针对集装箱港口自动装卸集装箱设计了一套纯视觉的定位系统，在适应港口环境条件中成像材料、固定材料、防护材料的要求基础上，实现了集装箱和起升机构吊具的分层级定位。该定位系统可以有效提高集装箱港口的装卸自动化程度，提高港口作业的安全性，降低作业成本，并提高港口的吞吐量和贸易流通效率。当前本课题设计开发的样机系统已部署在国内某港口的龙门式起重机上。

[1] 赤木. 上海国际航运中心集装箱枢纽港“中心两翼”核心港口回顾与展望[J]. 中国港口, 2023, No.369(11-13).
[2] 毕成成. 基于集装箱港口绩效指数的我国集装箱港口综合服务水平比较[J/OL]. 集装箱化, 2023, 34(1-5). DOI:10.13340/j.cont.2023.01.001.
[3] MI C, HUANG Y, FU C, et al. Vision-based measurement: actualities and developing trends in automated container terminals[J].IEEE Instrumentation and Measurement Magazine, 2021, 24(4): 65-76.
[4] 王康, 杨勇生, 姚海庆. 基于反射靶标的激光 SLAM 定位方法在港口无人车辆上的应用[J]. 全球定位系统, 2021, 46(65-72).
[5] 徐家, 龚国兴, 窦琴. 基于激光雷达的集装箱堆场智能安全防护与辅助作业系统[J]. 港口科技, 2020, No.172(1-5+27).
[6] 张冒, 程文明, 覃琴. 基于激光雷达测量技术的集装箱定位系统[J/OL]. 电气传动, 2022(75-80). DOI: 10.19457/j.1001-2095.dqcd22761.
[7] 杨育青. 自动化集装箱轮胎吊定位系统关键技术与应用[J]. 港工技术, 2021.
[8] 王亮, 刘诤轩. 融合双目视觉和 2D 激光雷达的室外定位方法[J/OL]. 控制与决策, 2021. DOI: 10.13195/j.kzyjc.2021.1899.
[9] 王家毅. 集卡防吊起系统在轮胎吊上的应用[J]. 港口科技, 2020(15-17+44).
[10] YOON H J, HWANG Y C, CHA E Y. Real-time container position estimation method using stereo vision for container auto-landing system[C]//International Conference on Control Automation & Systems. 2010.
[11] 张军, 程文明, 杜润, 等. 基于视频流的起重机吊具与集装箱对位研究[J]. 起重运输机械, 2020(13): 6.
[12] 丁小虎, 陈宁. 港口集装箱自主识别定位技术[J]. 集美大学学报：自然科学版, 2019, 24(4):9.
[13] PARK D, PARK S, BYUN S, et al. Container chassis alignment and measurement based on vision for loading and unloading containers automatically[C]//Hybrid Information Technology, 2006. ICHIT ’06. International Conference on. 2006.
[14] 宓为建, 张志伟, 宓超. 基于机器视觉的集装箱锁孔识别算法研究[J]. 中国工程机械学报, 2016, 14(5): 4.
[15] 谢孟添, 刁云峰, 程文明, 等. 基于 U-net 和 YOLOv4-tiny 的锁孔中心定位算法[J]. 起重运输机械, 2021(23): 6.
[16] 张军, 刁云峰, 程文明, 等. 基于视频流的集装箱锁孔追踪及中心定位[J]. 计算机应用, 2019, 39(S02): 5
[17] HUANG Q, ZHANG Y, HUANG Y, et al. Two-stage container keyhole location algorithm based on optimized SSD and adaptive threshold[J]. Journal of computational methods in sciences and engineering, 2022, 22(5): 1559-1571.
[18] 宓超, 黄世凤, 张钰洁, 等. 自动化集装箱码头装卸目标三维测量系统设计[J/OL]. 电子测量与仪器学报, 2023, 37(160-170). DOI: 10.13382/j.jemi.B2205887.
[19] YUAN Q, TENG X, TU S, et al. Atmospheric fine particles in a typical coastal port of Yangtze River Delta[J]. Journal of Environmental Sciences, 2020.
[20] WANG Y, HUANG T Z, ZHAO X L, et al. Rain streaks removal for single image via directional total variation regularization[C]//2019 IEEE International Conference on Image Processing (ICIP). 2019.
[21] CHOI L K, YOU J, BOVIK A C. Referenceless prediction of perceptual fog density and perceptual image defogging[J]. IEEE transactions on image processing, 2015, 24(11): 3888-3901.
[22] TRON R, VIDAL R. Distributed computer vision algorithms[C]//IEEE Conference on Computer Vision &Pattern Recognition. 2011: 32-45.
[23] 李孟涛, 吴翔, 蒋海新. 防打保龄系统在自动化轮胎式集装箱起重机中的应用[J]. 起重运输机械, 2022, No.599(71-74).
[24] 胡国伟. 水利水电工程金属结构腐蚀研究[J]. 山西水利, 2022, No.308(67-68).
[25] HARADA T, TERANISHI N, WATANABE T, et al. High-exposure-durability, high-quantumefficiency backside-illuminated soft-X-ray CMOS sensor[J]. Applied physics express, 2020, 13(1): 16502-.
[26] SOURI K, CHAE Y, MAKINWA K A A. A CMOS temperature sensor with a voltage-calibrated inaccuracy of ± 0.15°C (3σ) from ―55°C to 125°C[J]. IEEE journal of solid-state circuits, 2013, 48(1): 292-301.
[27] PRADO R, BEOBIDE G, MARCAIDE A, et al. Development of multifunctional sol–gel coatings: Anti-reflection coatings with enhanced self-cleaning capacity[J]. Solar energy materials and solar cells, 2010, 94(6): 1081-1088.
[28] SUN X, XU X, SONG G, et al. Preparation of MgF2/SiO2 coating with broadband antireflective coating by using sol–gel combined with electron beam evaporation[J]. Optical Materials, 2020, 101: 109739-.
[29] GAO Z, ZHAI G, YANG X. Stereoscopic 3D geometric distortions analyzed from the viewer’s point of view[J]. PloS one, 2020, 15(10): e0240661-e0240661.
[30] AL-BANDAWI H, DENG G. Classification of image distortion based on the generalized Benford’s law[J]. Multimedia tools and applications, 2019, 78(18): 25611-25628.
[31] ZHANG Z. A flexible new technique for camera calibration[J]. IEEE transactions on pattern analysis and machine intelligence, 2000, 22(11): 1330-1334.
[32] HARTLEY R. Multiple view geometry in computer vision[M]. 2nd ed. Cambridge, UK: Cambridge University Press, 2003.
[33] NISTER, D. An efficient solution to the five-point relative pose problem[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence, 2004, 26(6): 756-770.
[34] TANG F, ADAM L, SI B. Group feature selection with multiclass support vector machine[M]. Elsevier B.V., 2018.
[35] QI R, LI X, ZHANG Y, et al. Multi-classification algorithm for human motion recognition based on IR-UWB radar[J]. IEEE sensors journal, 2020, 20(21): 12848-12858.
[36] REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection[C]//Computer Vision & Pattern Recognition. 2016.
[37] SU S, FU Y, LAI L. Nut detection using single shot multibox detector and AlexNet[J]. Zhongguo Keji Lunwen = China Sciencepaper, 2018(4): 414-.
[38] ZHANG L, CHENG J, LIU J, et al. Unsupervised ship detection in SAR images using superpixels and CSPNet[J]. IEEE geoscience and remote sensing letters, 2023, 20: 1-.
[39] GITHUB. open-mmlab/mmyolo: OpenMMLab YOLO seriestoolbox and benchmark[Z]. 2023.
[40] XU D, HE S, ZHOU J, et al. Goos–Hänchen effect enabled optical differential operation and image edge detection[J]. Applied physics letters, 2020, 116(21): 211103-.
[41] CHUM O, MATAS J. Optimal randomized RANSAC[J]. IEEE transactions on pattern analysis and machine intelligence, 2008, 30(8): 1472-1482.
[42] BARATH D, MATAS J. Graph-cut RANSAC[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 2018: 6733-6741.
[43] VARALAKSHMI A, SANTHOSH KUMAR S, SHANMUGAPRIYA M M, et al. Markers location monitoring on images from an infrared camera using optimal fuzzy inference system[J]. International journal of fuzzy systems, 2023, 25(2): 731-742.
[44] CAHILL-ROWLEY K, ROSE J. Temporal spatial reach parameters derived from inertial sensors: Comparison to 3D marker-based motion capture[J]. Journal of biomechanics, 2017, 52: 11-16.
[45] FARIA S, LIMA J, COSTA P. Sensor fusion for mobile robot localization using extended Kalman filter, UWB ToF and ArUco markers[M]//Communications in Computer and Information Science: Optimization, Learning Algorithms and Applications. Cham: Springer International Publishing: 235-250.
[46] YUAN T, SONG Y W, KRAAN G A, et al. Identify finger rotation angles with ArUco markers and action cameras[J]. Journal of computing and information science in engineering, 2022, 22(3).