S3-Net: A Fast and Lightweight Video Scene Understanding Network by Single-shot Segmentation

Real-time understanding in video is crucial in variousAI applications such as autonomous driving. This workpresents a fastsingle-shot segmentationstrategy for videoscene understanding. The proposed net, called S3-Net,quickly locates and segmentstarget sub-scenes, meanwhileextractsstructured time-series semantic featuresas inputsto an LSTM-based spatio-temporal model. Utilizing ten-sorization and quantization techniques, S3-Net is intendedto be lightweight for edge computing. Experiments usingCityScapes, UCF11, HMDB51 and MOMENTS datasetsdemonstrate that the proposed S3-Net achieves an accuracyimprovement of8.1%versus the 3D-CNN based approachon UCF11, a storage reduction of6.9×and an inferencespeed of22.8FPS on CityScapes with a GTX1080Ti GPU