RTCSR: Zero-latency Aware Super-resolution for WebRTC Mobile Video Streaming

Yu, Qian1; Li, Qing2; He, Rui1; Shi, Wanxin3; Jiang, Yong3

10.1145/3609395.3610601

2023

SIGCOMM Workshop on Emerging Multimedia Systems (EMS)

PROCEEDINGS OF THE 2023 WORKSHOP ON EMERGING MULTIMEDIA SYSTEMS, EMS 2023

SEP 10, 2023

null,New York,NY

1601 Broadway, 10th Floor, NEW YORK, NY, UNITED STATES

ASSOC COMPUTING MACHINERY

The real-time video streaming market is currently undergoing rapid development. Various existing methods aim to optimize network bandwidth utilization by adjusting the sending rate and implementing forward error correction techniques. Meanwhile, other methods enhance the quality of video streams using frame-level super-resolution (SR), leveraging the computational power on the receiving end, which often consumes a lot of time and increases the end-to-end delay of a real-time video stream. To this end, we proposed a real-time video stream enhancement scheme based on patch-level super-resolution. The approach adaptively selects the locations of super-resolution patches based on the time interval between video frame receiving and rendering. Additionally, the quality benefit of SR patches continuously would be transferred to subsequent video frames during the decoding and upscaling process. We evaluate the performance of our scheme on several commercial mobile devices with real network traces and construct an open-source real-time video streaming system. Overall, our scheme can improve the PSNR of real-time video streaming by 2.3dB.

real-time video streaming video codec super-resolution

第一

英语

EI ; CPCI-S

Computer Science

Computer Science, Interdisciplinary Applications ; Computer Science, Theory & Methods

WOS:001141308100009

Web of Science

1.SUSTech, Peng Cheng Laboratory, Shenzhen, China
2.Peng Cheng Laboratory, Shenzhen, China
3.International Graduate School, Tsinghua University, Shenzhen, China

Yu, Qian,Li, Qing,He, Rui,et al. RTCSR: Zero-latency Aware Super-resolution for WebRTC Mobile Video Streaming[C]. 1601 Broadway, 10th Floor, NEW YORK, NY, UNITED STATES:ASSOC COMPUTING MACHINERY,2023.