Watching a small portion could be as good as watching all: Towards efficient video classification

Fan, Hehe1,2; Xu, Zhongwen2; Zhu, Linchao1,2; Yan, Chenggang3; Ge, Jianjun4; Yang, Yi1,2

2018

1045-0823

27th International Joint Conference on Artificial Intelligence, IJCAI 2018

2018-July

705-711

Stockholm, Sweden

International Joint Conferences on Artificial Intelligence

We aim to significantly reduce the computational cost for classification of temporally untrimmed videos while retaining similar accuracy. Existing video classification methods sample frames with a predefined frequency over entire video. Differently, we propose an end-to-end deep reinforcement approach which enables an agent to classify videos by watching a very small portion of frames like what we do. We make two main contributions. First, information is not equally distributed in video frames along time. An agent needs to watch more carefully when a clip is informative and skip the frames if they are redundant or irrelevant. The proposed approach enables the agent to adapt sampling rate to video content and skip most of the frames without the loss of information. Second, in order to have a confident decision, the number of frames that should be watched by an agent varies greatly from one video to another. We incorporate an adaptive stop network to measure confidence score and generate timely trigger to stop the agent watching videos, which improves efficiency without loss of accuracy. Our approach reduces the computational cost significantly for the large-scale YouTube-8M dataset, while the accuracy remains the same.
© 2018 International Joint Conferences on Artificial Intelligence. All right reserved.

第一

EI

20184406016240

Artificial intelligence ; Classification (of information)

Information Theory and Signal Processing:716.1 ; Artificial Intelligence:723.4

EV Compendex

1.SUSTech-UTS Joint Centre of CIS, Southern University of Science and Technology, United Kingdom
2.Centre for Artificial Intelligence, University of Technology Sydney, Australia
3.Institute of Information and Control, Hangzhou Dianzi University, China
4.Information Science Academy, CETC, China

Fan, Hehe,Xu, Zhongwen,Zhu, Linchao,et al. Watching a small portion could be as good as watching all: Towards efficient video classification[C]:International Joint Conferences on Artificial Intelligence,2018:705-711.