NPR: Nocturnal Place Recognition Using Nighttime Translation in Large-Scale Training Procedures

Bingxi Liu1,2; Yujie Fu3,4; Feng Lu2,5; Jinqiang Cui2; Yihong Wu3,4; Hong Zhang6

2024

10.1109/JSTSP.2024.3403247

IEEE Journal of Selected Topics in Signal Processing   影响因子和分区

1941-0484

Visual Place Recognition (VPR) is critical in intelligent robotics and computer vision. It involves retrieving similar database images based on a query photo from an extensive collection of known images. In real-world applications, this task encounters challenges when dealing with extreme illumination changes caused by nighttime query images. However, a large-scale training set with day-night correspondence for VPR remains absent. To address this challenge, we propose a novel pipeline that divides the general VPR into distinct domains of day and night, subsequently conquering Nocturnal Place Recognition (NPR). Specifically, we first establish a day-night street scene dataset named NightStreet and use it to train an unpaired image-to-image translation model. Then, we utilize this model to process existing large-scale VPR datasets, generate the night version of VPR datasets, and demonstrate how to combine them with two popular VPR pipelines. Finally, we introduce a divide-and-conquer VPR framework designed to solve the degradation of NPR during daytime conditions. We provide comprehensive explanations at theoretical, experimental, and application levels. Under our framework, the performance of previous methods can be significantly improved on two public datasets, including the top-ranked method. Datasets, code, and trained models are available for research at https://github.com/BinuxLiu/npr.

EI

第一

1.Department of Electronic and Electrical Engineering, Southern University of Science and Technology, Shenzhen, China
2.Peng Cheng Laboratory, Shenzhen, China
3.State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing, China
4.School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China
5.Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China
6.Shenzhen Key Laboratory of Robotics and Computer Vision, the Department of Electronic and Electrical Engineering, Southern University of Science and Technology, Shenzhen, China

Bingxi Liu,Yujie Fu,Feng Lu,et al. NPR: Nocturnal Place Recognition Using Nighttime Translation in Large-Scale Training Procedures[J]. IEEE Journal of Selected Topics in Signal Processing,2024,PP(99).

Bingxi Liu,Yujie Fu,Feng Lu,Jinqiang Cui,Yihong Wu,&Hong Zhang.(2024).NPR: Nocturnal Place Recognition Using Nighttime Translation in Large-Scale Training Procedures.IEEE Journal of Selected Topics in Signal Processing,PP(99).

Bingxi Liu,et al."NPR: Nocturnal Place Recognition Using Nighttime Translation in Large-Scale Training Procedures".IEEE Journal of Selected Topics in Signal Processing PP.99(2024).