MacSR: Macroblock-aware Lightweight Video Super-Resolution

He, Rui1,2; Li, Qing1; Yu, Qian1,2; Yuan, Zhenhui3; Shi, Wanxin4; Lv, Jianhui1; Han, Yi5

10.1109/DCC55655.2023.00061

2023

Data Compression Conference (DCC)

1068-0314

2023 DATA COMPRESSION CONFERENCE, DCC

MAR 21-24, 2023

null,Snowbird,UT

10662 LOS VAQUEROS CIRCLE, PO BOX 3014, LOS ALAMITOS, CA 90720-1264 USA

IEEE COMPUTER SOC

["The mobile video quality can be improved by video super-resolution (SR) especially when bandwidth is limited. To achieve real-time SR, the latest work, ClassSR (CVPR 19), divides frames into equal-size image blocks (IBs), and different-complexity SR models are used respectively to reduce the computational burden.","We propose a lightweight IB-based SR scheme, MacSR, for on-demand videos. In Figure 1, the server first runs the time modeling algorithm based on decoding information to estimate video quality (i.e., PSNR) improvement of each 16x16 IB SR (i.e., SR impact). Then, for different types of IBs, we use forward propagating and intra-IB sorting to select IBs and generate corresponding sets (i.e., intra and inter lists). Finally, we use the block searching algorithm to search the appropriate inter-IB and intra-IB proportions according to video quality requirements. The algorithm finally obtains the SR IB indexes saved in patch lists. On the client side, we redesign the decoder which super-resolves the selected IBs in patch lists, while other IBs rely on the high-resolution reference frame queue for decoding. In this way, the terminal can output high-quality videos with low computation.","[GRAPHICS]","We measure the latency of decoding and SR on the Xeon E5-2650 CPU to quantify the acceleration, compared to per-frame SR in six 240p videos (three types). In the IB-based SR mode, all schemes use the same SR model, and the input frame is divided into 16x16 blocks. The adopted SR model is a variation of MDSR (CVPRW 17). In Figure 2, MacSR achieves 32%, 23.2%, and 24% acceleration compared with Nemo (MobiCom 20). The acceleration further increases to 48.75%,39.2%, and 105.5%, compared with Fast (CVPRW 17). MacSR maintains robustness with video content changes. The average video quality of MacSR is 0.25dB higher than Fast. The average extra bandwidth brought by patch lists is less than 3% of video chunks."]

其他

英语

CPCI-S

NSFC[61972189]

Computer Science ; Imaging Science & Photographic Technology

Computer Science, Software Engineering ; Computer Science, Theory & Methods ; Imaging Science & Photographic Technology

WOS:001008252400042

Web of Science

1.Peng Cheng Lab, Shenzhen, Peoples R China
2.SUSTech, Shenzhen, Peoples R China
3.Northumbria Univ, Newcastle Upon Tyne, Tyne & Wear, England
4.Tsinghua SIGS, Shenzhen, Peoples R China
5.WUT, Wuhan, Peoples R China

He, Rui,Li, Qing,Yu, Qian,et al. MacSR: Macroblock-aware Lightweight Video Super-Resolution[C]. 10662 LOS VAQUEROS CIRCLE, PO BOX 3014, LOS ALAMITOS, CA 90720-1264 USA:IEEE COMPUTER SOC,2023.