基于嵌入式的车载环视系统

先进辅助驾驶系统旨在使用一系列科技手段辅助驾驶员，增加驾驶可靠性，减少安全事故的发生。而全景环视算法作为辅助驾驶技术的一种，已经在现今的车辆上有了广泛的应用。它通过在汽车的四周装配多个鱼眼相机，并使用算法对鱼眼相机获取的图像进行处理，最终以多个视角的图像展示在车载显示器上，可以增加驾驶员视野，消除视野盲区。目前的环视算法主要依赖FPGA（Field-Programmable Gate Array，现场可编程门阵列）和GPU（Graphics Processing Unit，图形处理器）。本文设计了一套适用于低算力嵌入式平台的三维环视算法。在图像预处理阶段，利用提取ROI （Region of Interest，感兴趣区域），HSV颜色筛选和形态学处理等方法获取特征点，无需手动选取，提高了在图像配准阶段的效率，并且降低了操作难度。此外，由于二维平面投影存在物体距离相机越远，拉伸越严重的弊端，本文中设计了一个碗状的三维模型用作投影曲面，大大消除了二维平面投影带来的拉伸和变形，使投影画面更加立体，同时扩大了驾驶员的视野范围。算法优化方面，采用查找表，并行重映射和使用SIMD（Single Instruction Multiple Data，单指令多数据）进行硬件加速等方法大幅减少计算耗时。不同于依靠高算力的GPU和FPGA平台的算法，本文中的算法仅依靠嵌入式平台即可流畅运行，对算力要求较小，三维效果较好，可视范围大。

ADAS (Advanced driver assistance systems) aims to assist drivers through a series of technical means, increasing driving safety and reducing safety accidents. As a kind of assisted driving technology, the 3D surround view algorithm has been widely used in today's vehicles. It installs multiple fisheye cameras around the body and uses algorithms to process the images collected by the fisheye cameras, and finally displays images from multiple viewing angles on the vehicle monitor, which can increase the driver's field of vision and eliminate blind spots. The current 3D surround view algorithms mainly rely on FPGA (Field-Programmable Gate Array) and GPU (Graphics Processing Unit). This paper proposes a 3D surround view algorithm suitable for embedded platforms with low computing power. The feature points are obtained by extracting the ROI (Region of Interest), HSV color screening and morphological processing in the image preprocessing stage, without manual selection, which improves the efficiency of image registration and reduces the difficulty of operation. In addition, the two-dimensional plane projection has the disadvantage that the farther the object is from the camera, the more serious the stretching is. Therefore, a bowl-shaped three-dimensional model is designed as a projection surface in this paper, which greatly eliminates the stretching and deformation caused by the two-dimensional plane projection and makes the projection screen more stereoscopic, while expanding the driver's field of vision. In terms of algorithm optimization, the use of LUT (Look-Up Tables), parallel remapping and hardware acceleration using SIMD (Single Instruction Multiple Data) greatly reduces the computational time. The algorithm in this paper can run smoothly only by relying on the embedded platform, and has large visual range.

[1] 铁 建 . 360 ° 环 视 系 统 成 功 进 入 车 载 前 装 市 场 [J]. 军民两用技术与产品，2017(3):23-23.
[2] 边宁，赵保华，赖锋等.基于高速公路的半自动驾驶辅助系统的开发与应用 [J]. 汽车安全与节能学报, 2017, 8(2): 149-156.
[3] 刘如筑，林贤捷，林宁. 浅谈汽车高级驾驶辅助系统(ADAS)的原理与应用[J]. 工程技术全文版, 2017(2): 274-275.
[4] CLANTON J M, BEVLY D M, HODEL A S. A Low-Cost Solution for An Integrated Multisensor Lane Departure Warning System[J]. IEEE Transactions on Intelligent Transportation Systems, 2009, 10(1): 47-59.
[5] "ABI Research Predicts Growth for Advanced Driver Assistance Systems Market in China after 2017, Driving 36% CAGR Revenue Growth through 2020.”[Online]. Available:https://www.abiresearch.com/press/abi-research-predicts-growth-advanced-driver-assis/. [Accessed: 15-May-2017].
[6] CHEN YY, TU YY, and CHEN YS, “An Embedded System for Vehicle Surro￾unding Monitoring,” Proc. 2nd International Conference on Power Electronics and Intelligent Transportation System, 2:92-95, 2009.
[7] LIU YC, LIN KY, and CHEN YS, “Bird’s-Eye View Vision System for Vehicl￾e Surrounding Monitoring,” Proc. 2nd International Conference on Robot Visio￾n, 207-218, 2008.
[8] NOH, JOO H, SONG, et al. AVM Method and Apparatus for Assessing an Image Match of an Around View Image for an Around View Monitor System[P].：KR20170111504,2017-10-12.
[9] SHIMIZU S, KAWAI J, YAMADA H. Wraparound View System for Motor Vehicles[J]. Fujitsu Scientific & Technical Journal, 2010, 46(1): 95-102.
[10] Anonymous. Business News; Fujitsu 360-Degree Wrap-Around Video Imaging Technology Toolset Ready for Integration with Latest Graphics SoCs[J]. Computers, Networks & Communications,2010.
[11] ZHANG BY, IBRAHIM P, VIKRAM A, et al. Method, apparatus and system for processing a display from a surround view camera solution[P]. ：US9533618,2017-01-03.
[12] Anonymous. Renesas to Release SH7777 SoC for Dashboard-Mounted Car Navigation[J]. Wireless News,2009
[13] MATTSON P, KIM D, KIM Y. “Generalized image warping using enhanced lookup tables,” Int. J. Imaging Syst. Technol. v9. 475-483.
[14] OH S, KIM G, “FPGA-based fast image warping with data-parallelization schemes,” Consumer Electronics, IEEE Trans., vol.54, no.4, pp.2053-2059, Nov. 2008.
[15] MOHAN A, SENAPATHI A K, SHANKAR R, et al. “Hardware Acceleration of Real Time Fish Eye Correction on FPGA,” NCIS-2009, Manipal.
[16] CHEN L, ZHANG M, WANG B, et al. “Real-Time FPGA-Based Panoramic Unrolling of High-Resolution Catadioptric Omnidirectional Images,” ICMTMA '09. Int. Conf. on, vol.1, no., pp.502-505, 11-12 April 2009.
[17] 邹超洋. 基于多摄像头全景图像拼接的实时视频监控技术研究[D]. 华南理工大学2014：41-54.
[18] 鲍千千. 基于可变曲面投影的汽车全景系统算法研究[D]. 燕山大学，2015：41-54.
[19] 江龙. 360°泊车辅助系统全景成像的研究[D].哈尔滨工业大学,2017.
[20] 连猛. 车联网智能车载终端的设计与实现[D]. 中北大学，2015：50-91.
[21] 陈泽茂. 基于全景视觉的汽车安全驾驶辅助系统的平台设计与实现[D]. 华南理工大学,2014.
[22] 汽车环视技术发展趋势浅析[J].单片机与嵌入式系统应用,2016,16(12):85.
[23] AUYSAKUL J, XU H, "Fish-eye image of hemi-cylinder unwrapping plane based on a flexible technique" in Fluid Power and Mechatronics (FPM), 2015 International Conference on, 2015, pp. 821–824.
[24] AUYSAKUL J, XU H, ZHAO W, "Development of Hemi-Cylinder Plane for Panorama View in Around View Monitor Applications" in Computational Intelligence and Applications (ICCIA), 2016 International Conference on, 2016, pp. 26 –30.
[25] SU, LEE C, LI M, et al. "3D AVM system for automotive applications," 2015 10th International Conference on Information, Communications and Signal Processing (ICICS), Singapore, 2015, pp. 1-5, doi: 10.1109/ICICS.2015.7459915.
[26] JEON, PARK G, LEE J, et al. "A Memory-Efficient Architecture of Full HD Around View Monitor Systems," in IEEE Transactions on Intelligent Transportation Systems, vol. 15, no. 6, pp. 2683-2695, Dec. 2014, doi: 10.1109/TITS.2014.2325215.
[27] MENG H, YUAN F, XU Y, "Embedded GPU 3D Panoramic Viewing System Based on Virtual Camera Roaming 3D Environment," in IEEE Access, vol. 7, pp. 130156 -130167, 2019, doi: 10.1109/ACCESS.2019.2938975.
[28] FENG S, JIANG Y, HUANG H, et al. "Research on Vehicle 3D Panoramic Model and Texture Mapping Algorithm Based on Low Distortion and Uniform Field of View," 2018 IEEE International Conference on Robotics and Biomimetics (ROBIO), Kuala Lumpur, Malaysia, 2018, pp. 450-455, doi: 10.1109/ROBIO.2018.8665150.
[29] 甘向昕.双目视觉测距在自动泊车系统中的应用[J].现代信息科技,2020,(1):52-53.
[30] 周薇.基于机器视觉的摄像机标定方法的研究 [J]. 电子元器件应用 , 2008(11): 70-72.
[31] 吴火森.鱼眼相机图像处理及校正算法研究与实现[D].西安电子科技大学,2015.
[32] ISO T, KURAKAKE M. FISH EYE CAMERA AND CALIBRATION METHOD IN THE FISH EYE CAMERA[P]. ：JP2005258953, 2005- 9-22.
[33] 李崇辉，杜兰，韩建，等. 圆形目标的鱼眼相机成像模型推导[J]. 河南科学， 2013(7)975-979.
[34] JUHO K, SAMI B. A generic camera model and calibration method for conventional, wide-angle, and fisheye lenses. [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2006,28(8).
[35] CHENG R, ZHAO Y, LI Z, et al. “Panorama parking assistant system with improved particle swarm optimization method,” J. Electron. Imaging, vol. 22, no. 4, pp. 41123, Dec. 2013.
[36] BRADSKI G, KAEHLER A, Learning OpenCV: Computer Vision with the OpenCV Library. O’Reilly Media, Inc., 2008.
[37] SONG T, CHU G Y, HOU P G, et al. Fisheye Camera Calibration Algorithm Optimized by Centroid Method[J]. Acta Photonica Sinica, 2016, 45(5): 512005.
[38] GUO XL, CHEN RJ, WANG JQ. Fast fisheye camera calibration methodusing￾stereoscopic calibration board[J]. Journal of Harbin Engineering University, 2016,37(11): 1594-1599.
[39] 徐静，刘延芳，唐中海．图像配准与角点提取算法研究[J]．长春工程学院学报(自然科学版)，2017，18(04)：104-107．
[40] ZHANG BY, IBRAHIM P, VIKRAM A, et al. Method, apparatus and system for processing a display from a surround view camera solution[P]. ：US9533618,2017-01-03.
[41] HEDI A, LONČARIĆ S. A System for Vehicle Surround View[J]. IFAC Proceedings Volumes,2012,45(22).
[42] SZELISKI R, Computer vision: algorithms and applications, Springer Science & Business Media, 2010.
[43] 刘 冬 , 秦 瑞 , 陈 曦 , 李 庆 .3D 车 载 环 视 全 景 生 成 方 法 [J]. 计 算 机 科 学 ,2017, 44(04):302-305.
[44] ISHII Y, ASARI K, HONGO H, et al. “A Practical Calibration Method for Top View Image Generation,” International Conference on Consumer Electronics, 1-2, 2008.
[45] KANO H, ASARI K, ISHII Y, et al. “Precise Top View Image Generation without Global Metric Information,” IEICE Transactions on Information and Systems, E91D(7):1893-1898, July 2008.
[46] LI S and HAI Y, “Easy Calibration of a Blind-Spot-Free Fisheye Camera System Using a Scene of a Parking Space,” IEEE Transactions on Intelligent Transportation Systems, 12(1):232-242, March 2011.
[47] LIN CC and WANG MS, “A Vision Based Top-View Transformation Model for a Vehicle Parking Assistant,” Sensors, 12:4431-4446, 2012.
[48] BROWN M and LOWE DG, “Automatic panoramic im-age stitching using invariant features,” International journal of computer vision, vol. 74, no. 1, pp. 59–73,2007.
[49] SELISKI R and SHUM HY, “Creating full view panoramic image mosaics and environment maps,” Annual conference on Computer graphics and interactivetechniques. ACM Press/Addison-Wesley PublishingCo., 1997, pp. 251–258.
[50] GAO J, KIM SJ and BROWN MS, “Constructing image panoramas using dualhomography warping,” IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2011, pp. 49–56.
[51] LIN CC, PANKANTI SU, and ARAVKIN A, “Adaptive as-natural-as-possible image stitching,” IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2015, pp. 1155–1163.
[52] 刘保权,刘学慧,吴恩华.基于 GPU 的实时深度图像前向映射绘制算法[J].软件学报,2007(06):1531-1542.