基于双目视觉的无人机着陆区识别技术研究

  无人机的自主着陆是无人机领域的重要研究方向之一，目前基于视觉的无人机自主着陆主要是针对合作目标来进行定点着陆，而如何在无合作目标时自主识别出地面中的可着陆区域并保证着陆区的安全性便成了一大难点。本文基于双目立体视觉，结合语义分割与立体匹配技术，使得无人机可以在无合作目标的场景中自主寻找优选着陆区。具体内容如下：
    本文首先提出使用OCR模块和HRNet_W18结合的语义分割模型来作为无人机正下方区域的地形分类模型，并基于DLRSD数据集完成了模型的训练与验证，分类精度Acc达到86.53%，其中关键地形类别的分类准确率为：草坪80.66%，道路91.86%，水域98.18%，树木84.3%，沙地90.33%，建筑87.22%，且单张分辨率为256×256的图像预测时间约为118ms，符合实时性的要求。同时针对较难分类的水域图像，本文利用1485张低分辨率的多水域场景的自建数据样本对模型进行测试，分类完全正确的图片数量为1477张，占比为99.46%。实验结果表明，本文所提出的地形分类方法可满足高空场景的实时垂直投影所拍摄的地面地形分类要求，实现初步筛除掉危险着陆地形，并筛选出常见的可着陆地形如道路、草坪、沙地等。
  
  在地形分类的基础上，本文基于传统立体匹配算法SGBM并针对本文应用场景提出了改进后的SGBM-A算法，首先对实验所用的ZED双目相机进行了标定与极线校正，标定的重投影误差为0.17pixels。其次基于双目相机系统的内外参数完成了双目图像的立体匹配和深度信息重建，该算法在本文的实际场景中的深度估计误差小于0.5%，单张大小为2208×1242的图像深度信息重建耗时约为326ms。之后基于深度信息设计并进行了平整区判定实验，最后基于平整区判定结果筛选出最大连通平整区，并对该区域基于深度信息结合数学三角函数公式来进行坡度计算，若该最大连通平整区的面积与坡度符合要求，则判定该区域为优选着陆区，并定义该区域的内接圆圆心则为优选着陆点。本文所提出的着陆区识别方法的的平均选址正确率为92.50%。实验结果证明，本文所提出的着陆区识别方法能够高精度实现无人机垂域尺度的优选着陆区选取。
 

[1]唐辉平, 陆海博,张爱东,等. 基于视觉识别的无人机自主降落无人艇的方法及系统[Z]. 2021.
[2] SCHARSTEIND,SZELISKIR,ZABIHR. ATaxonomyandEvaluationofDenseTwo-Frame Stereo CorrespondenceAlgorithms[C]//StereoandMulti-BaselineVision, 2001.(SMBV2001). Proceedings. IEEE Workshop on. 2002.
[3]席志鹏. 无人机自主飞行若干关键问题研究[D]. 浙江大学,2019.
[4] LIN S, GARRATT M A, LAMBERT A J. Monocular vision-based real-time target recognition and tracking for autonomously landing an UAV in a cluttered shipboard environment[J]. Autonomous Robots, 2017, 41(4): 1-21.
[5]徐焕太. 基于双目视觉的多旋翼无人机自主降落定位方法研究[D]. 哈尔滨理工大学, 2018.
[6]杜晶,雷志辉,周翔.基于红外探测技术的无人机视觉引导助降系统[J].计算机工程,2013, 39: 237-241.
[7]项立,王全辉,陈文霞. 基于红外信标的植保无人机助降系统研究[J]. 现代计算机,2021, 27(34): 6.
[8] CABRERA-PONCE A A, MARTINEZ-CARRANZA J. A vision-based approach for autonomous landing[C]//2017 Workshop on Research, Education and Development of Unmanned Aerial Systems (RED UAS). 2017.
[9] GARCIA-GARCIA A, ORTS-ESCOLANO S, OPREA S, et al. A Review on Deep Learning Techniques Applied to Semantic Segmentation[Z]. 2017.
[10] RONNEBERGER O, FISCHER P, BROX T. U-net: Convolutional networks for biomedical image segmentation[C]//Medical image computing and computer-assisted interventionMICCAI 2015: 18th international conference, Munich, Germany, October 5-9, 2015, proceedings, part III 18. Springer, 2015: 234-241.
[11] LONG J, SHELHAMER E, DARRELL T. Fully Convolutional Networks for Semantic Segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 39(4): 640-651.
[12] BEHERA T K, BAKSHI S, SA P K. Vegetation Extraction from UAV-based Aerial Images through Deep Learning[J]. Computers and Electronics in Agriculture, 2022.
[13] ZHAO H, SHI J, QI X, et al. Pyramid scene parsing network[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 2881-2890.
[14] HEK,ZHANGX,RENS,etal. Deepresiduallearning for image recognition[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 770-778.
[15] YUF,KOLTUNV. Multi-scale context aggregation by dilated convolutions[A]. 2015.
[16] 张娣. 基于双目视觉的道路场景语义分割技术研究[D]. 南京理工大学,2020.
[17] CORDTSM,OMRANM,RAMOSS,etal. TheCityscapesDatasetforSemanticUrbanScene Understanding[C]//Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2016.
[18] SUN Q, ZHANG R, CHEN L, et al. Semantic segmentation and path planning for orchards based on UAV images[J]. Computers and Electronics in Agriculture, 2022, 200: 107222.
[19] ROYAG,NAVABN,WACHINGERC. Concurrent spatial and channel ‘squeeze & excitation’in fully convolutional networks[C]//Medical Image Computing and Computer Assisted Intervention–MICCAI 2018: 21st International Conference, Granada, Spain, September 16-20, 2018, Proceedings, Part I. Springer, 2018: 421-429.
[20] CHENLC,PAPANDREOUG,SCHROFFF,etal. Rethinkingatrousconvolutionforsemantic image segmentation[A]. 2017.
[21] ALOMMZ,HASANM,YAKOPCICC,etal.Recurrentresidualconvolutionalneuralnetwork based on u-net (r2u-net) for medical image segmentation[A]. 2018.
[22] GIRISHA S, MM M P, VERMA U, et al. Semantic segmentation of UAV aerial videos using convolutional neural networks[C]//2019 IEEE Second International Conference on Artificial Intelligence and Knowledge Engineering (AIKE). IEEE, 2019: 21-27.
[23] 范满. 机器视觉在旋翼无人机应急降落选址应用研究[D]. 中国民用航空飞行学院,2022.
[24] YUAN Y, CHEN X, WANG J. Object-contextual representations for semantic segmentation [C]//Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part VI 16. Springer, 2020: 173-190.
[25] CHEN L C, PAPANDREOU G, SCHROFF F, et al. Rethinking Atrous Convolution for Semantic Image Segmentation[Z]. 2017.
[26] ILLIA,BOUZAACHANEK,ElHadajS,etal. Apixel-wise labelled dataset of Moroccan aircraft emergency landing sites for semantic segmentation applications[J/OL]. Data in Brief, 2024, 54: 110379. https://www.sciencedirect.com/science/article/pii/S2352340924003482. DOI: https://doi.org/10.1016/j.dib.2024.110379.
[27] GONÇALVESDN,JUNIORJM,CARRILHOAC,etal. Transformers for mapping burned areas in Brazilian Pantanal and Amazon with PlanetScope imagery[J]. International Journal of Applied Earth Observation and Geoinformation, 2023, 116: 103151.
[28] XIE E, WANG W, YU Z, et al. SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers[Z]. 2021.
[29] VASWANI A, SHAZEER N, PARMAR N, et al. Attention Is All You Need[J/OL]. CoRR, 2017, abs/1706.03762. http://arxiv.org/abs/1706.03762.
[30] YOON K J, KWEON I S. Adaptive support-weight approach for correspondence search[J]. IEEE transactions on pattern analysis and machine intelligence, 2006, 28(4): 650-656.
[31] SCHARSTEIND,SZELISKIR. Ataxonomy and evaluation of dense two-frame stereo correspondence algorithms[J]. International journal of computer vision, 2002, 47: 7-42.
[32] HIRSCHMüLLERH. Accurateand Efficient Stereo Processing by Semi-Global Matching and Mutual Information[J]. IEEE Computer Society, 2005.
[33] BOYKOVYY. Interactive graph cuts for optimal boundary & region segmentation of objects in n-d images[C]//Proc Eighth IEEE International Conference on Comput Vis. 2001.
[34] 李沛轩. 基于双目视觉的机翼形变测量技术研究[D]. 南方科技大学,2023.
[35] YAOG,CUIJ,DENGK,etal. RobustHarriscornermatching based on the quasi-homography transform and self-adaptive window for wide-baseline stereo images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 56(1): 559-574.
[36] HIRSCHMULLER H. Stereo processing by semiglobal matching and mutual information[J]. IEEE Transactions on pattern analysis and machine intelligence, 2007, 30(2): 328-341.
[37] GALLUP D, FRAHMJM,MORDOHAIP,etal. Real-time plane-sweeping stereo with multiple sweeping directions[C]//2007 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2007: 1-8.
[38] YAO Y, LUO Z, LI S, et al. Mvsnet: Depth inference for unstructured multi-view stereo[C]// Proceedings of the European conference on computer vision (ECCV). 2018: 767-783.
[39] YAOY,LUOZ,LIS,etal. Recurrentmvsnetforhigh-resolution multi-view stereo depth inference[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019: 5525-5534.
[40] CHEN R, HANS, XUJ, et al. Point-based multi-view stereo network[C]//Proceedings of the IEEE/CVF international conference on computer vision. 2019: 1538-1547.
[41] GU X, FAN Z, ZHU S, et al. Cascade cost volume for high-resolution multi-view stereo and stereo matching[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2020: 2495-2504.
[42] GUO X, YANG K, YANG W, et al. Group-Wise Correlation Stereo Network[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2020.
[43] CHANG J R, CHEN Y S. Pyramid stereo matching network[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 5410-5418.
[44] YUZ, GAOS. Fast-mvsnet: Sparse-to-dense multi-view stereo with learned propagation and gauss-newton refinement[C]//Proceedings of the IEEE/CVF conference on computervision and pattern recognition. 2020: 1949-1958.
[45] 陈利燕,林鸿,吴健华. 融合随机森林和超像素分割的建筑物自动提取[J/OL]. 测绘通报, 2021: 49-53. DOI: 10.13474/j.cnki.11-2246.2021.0042.
[46] 乔梦佳,王宇飞,赫晓慧,等.基于影像分割与SVM分类的城市建筑物提取研究[J/OL].信息技术,2018: 30-33+38. DOI: 10.13274/j.cnki.hdzj.2018.05.008.
[47] 李潇凡,王胜强,翁轩,等. 基于UNet深度学习算法的东海大型漂浮藻类遥感监测[J]. 光学学报,2021,41: 18-26.
[48] 张哲晗,方薇,杜丽丽,等. 基于编码-解码卷积神经网络的遥感图像语义分割[J]. 光学学报,2020, 40: 46-55.
[49] 任欣磊,王阳萍,杨景玉,等. 基于改进U-net的遥感影像建筑物提取[J]. 激光与光电子学进展,2019, 56: 195-202.
[50] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84-90.
[51] CLEVERT D A, UNTERTHINER T, HOCHREITER S. Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)[J]. Computer Science, 2015.
[52] IOFFE S, SZEGEDY C. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift[J]. JMLR.org, 2015.
[53] LINM,CHENQ,YANS. Networkinnetwork[A]. 2013.
[54] SZEGEDY C, LIU W, JIA Y, et al. Going Deeper with Convolutions[J]. IEEE Computer Society, 2014.
[55] HINTONGE,SRIVASTAVA N, KRIZHEVSKY A, et al. Improving neural networks by preventing co-adaptation of feature detectors[Z]. 2012: págs. 212-223.
[56] CHENY,WANGY,LUP,etal. Large-scalestructurefrommotionwithsemantic constraints of aerial images[C]//Chinese Conference on Pattern Recognition and Computer Vision (PRCV). Springer, 2018: 347-359.
[57] NIGAMI,HUANGC,RAMANAND. EnsembleKnowledgeTransfer for Semantic Segmentation[C]//IEEE Winter Conference on Applications of Computer Vision. 2018.
[58] LYU Y, VOSSELMANG, XIA GS,et al. UAVid: A semantic segmentation dataset for UAV imagery[J/OL]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 165: 108- 119. http://www.sciencedirect.com/science/article/pii/S0924271620301295. DOI: https://doi.org/ 10.1016/j.isprsjprs.2020.05.009.
[59] SHAOZ, YANGK,ZHOUW. Performance Evaluation of Single-Label and Multi-Label Remote Sensing Image Retrieval Using a Dense Labeling Dataset[J]. Remote Sensing, 2018(6).
[60] CHAUDHURI B, DEMIR B, CHAUDHURI S, et al. Multilabel Remote Sensing Image Retrieval Using a Semisupervised Graph-Theoretic Method[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(2): 1144-1158.
[61] YANGY,NEWSAMS. Bag-of-visual-words and spatial extensions for land-use classification [C]//ACM. 2010: 270.
[62] SUN K, XIAO B, LIU D, et al. Deep High-Resolution Representation Learning for Human Pose Estimation[C]//CVPR. 2019.
[63] XIAO B, WU H, WEI Y. Simple Baselines for Human Pose Estimation and Tracking[C]// European Conference on Computer Vision (ECCV). 2018.
[64] WANGJ,SUNK,CHENGT,etal. DeepHigh-Resolution Representation Learning for Visual Recognition[J]. TPAMI, 2019.
[65] WANGJ,SUNK,CHENGT,etal. DeepHigh-Resolution Representation Learning for Visual Recognition[Z]. 2019.
[66] LIU Y, CHU L, CHEN G, et al. PaddleSeg: A High-Efficient Development Toolkit for Image Segmentation[Z]. 2021.
[67] 邵军军. 基于双目立体视觉的行人检测与测距系统研究[D]. 兰州理工大学,2022.
[68] BOUGUETJY,PERONAP. CameraCalibration from Points and Lines in Dual-Space Geometry[Z]. 1998.
[69] MATTOCCIAS. Stereo vision: Algorithms and applications[J]. University of Bologna, 2013, 22.
[70] 何雨. 基于深度学习的室内环境感知方法研究[D]. 西安工业大学,2023.
[71] 李云廷. 基于立体视觉的三维精确测量方法研究[D]. 华中科技大学,2016.
[72] 耿英楠. 立体匹配技术的研究[D]. 吉林大学,2014.
[73] 夏珂. 垂直起降无人机视觉助降技术研究[D]. 南方科技大学,2022