Norm-guided Adaptive Visual Embedding for Zero-Shot Sketch-Based Image Retrieval

Wang，Wenjie1; Shi，Yufeng1; Chen，Shiming1; Peng，Qinmu1; Zheng，Feng3; You，Xinge1,2

2021

1045-0823

IJCAI International Joint Conference on Artificial Intelligence

1106-1112

Zero-shot sketch-based image retrieval (ZS-SBIR), which aims to retrieve photos with sketches under the zero-shot scenario, has shown extraordinary talents in real-world applications. Most existing methods leverage language models to generate class-prototypes and use them to arrange the locations of all categories in the common space for photos and sketches. Although great progress has been made, few of them consider whether such pre-defined prototypes are necessary for ZS-SBIR, where locations of unseen class samples in the embedding space are actually determined by visual appearance and a visual embedding actually performs better. To this end, we propose a novel Norm-guided Adaptive Visual Embedding (NAVE) model, for adaptively building the common space based on visual similarity instead of language-based pre-defined prototypes. To further enhance the representation quality of unseen classes for both photo and sketch modality, modality norm discrepancy and noisy label regularizer are jointly employed to measure and repair the modality bias of the learned common embedding. Experiments on two challenging datasets demonstrate the superiority of our NAVE over state-of-the-art competitors.

其他

英语

EI

National Natural Science Foundation of China[61772220];Science, Technology and Innovation Commission of Shenzhen Municipality[JCYJ20180305180637611];Science, Technology and Innovation Commission of Shenzhen Municipality[JCYJ20180305180804836];Science, Technology and Innovation Commission of Shenzhen Municipality[JSGG20180507182030600];

20220911735453

Artificial intelligence ; Embeddings ; Image retrieval ; Visual languages

Computer Programming Languages:723.1.1 ; Artificial Intelligence:723.4

2-s2.0-85120554331

Scopus

1.School of Electronic Information and Communication,Huazhong University of Science and Technology,
2.Shenzhen Research Institute,Huazhong University of Science and Technology,
3.Department of Computer Science and Engineering,Southern University of Science and Technology,

Wang，Wenjie,Shi，Yufeng,Chen，Shiming,et al. Norm-guided Adaptive Visual Embedding for Zero-Shot Sketch-Based Image Retrieval[C],2021:1106-1112.