CoroGraph: Bridging Cache Efficiency and Work Efficiency for Graph Algorithm Execution

Zhi, Xiangyu1; Yan, Xiao1; Tang, Bo1; Yin, Ziyao1; Zhu, Yanchao2; Zhou, Minqi2

2023-12-01

10.14778/3636218.3636240

PROCEEDINGS OF THE VLDB ENDOWMENT   影响因子和分区

2150-8097

Many systems are designed to run graph algorithms efficiently in memory but they achieve only cache efficiency or work efficiency. We tackle this fundamental trade-off in existing systems by designing CoroGraph, a system that attains both cache efficiency and work efficiency for in-memory graph processing. CoroGraph adopts a novel hybrid execution model, which generates update messages at vertex granularity to prioritize promising vertices for work efficiency, and commits updates at partition granularity to share data access for cache efficiency. To overlap the random memory access of graph algorithms with computation, CoroGraph extensively uses coroutine, i.e., a lightweight function in C++ that can yield and resume with low overhead, to prefetch the required data. A suite of designs are incorporated to reap the full benefits of coroutine, which include prefetch pipeline, cache-friendly graph format, and stop-free synchronization. We compare CoroGraph with five state-of-the-art graph algorithm systems via extensive experiments. The results show that CoroGraph yields shorter algorithm execution time than all baselines in 18 out of 20 cases, and its speedup over the best-performing baseline can be over 2x. Detailed profiling suggests that CoroGraph achieves both cache efficiency and work efficiency with a low memory stall and a small number of processed edges.

SCI

英语

第一 ; 通讯

Guangdong Basic and Applied Basic Research Foundation[2021A 1515110067] ; Shenzhen Fundamental Research Program[20220815112848002] ; Guangdong Provincial Key Laboratory[2020B121201001]

Computer Science

Computer Science, Information Systems ; Computer Science, Theory & Methods

WOS:001206935800022

ASSOC COMPUTING MACHINERY

Web of Science

1.Southern Univ Sci & Technol, Dept Comp Sci & Engn, Shenzhen, Peoples R China
2.Huawei Co, Gauss Dept, Shenzhen, Peoples R China

Zhi, Xiangyu,Yan, Xiao,Tang, Bo,et al. CoroGraph: Bridging Cache Efficiency and Work Efficiency for Graph Algorithm Execution[J]. PROCEEDINGS OF THE VLDB ENDOWMENT,2023,17(4).

Zhi, Xiangyu,Yan, Xiao,Tang, Bo,Yin, Ziyao,Zhu, Yanchao,&Zhou, Minqi.(2023).CoroGraph: Bridging Cache Efficiency and Work Efficiency for Graph Algorithm Execution.PROCEEDINGS OF THE VLDB ENDOWMENT,17(4).

Zhi, Xiangyu,et al."CoroGraph: Bridging Cache Efficiency and Work Efficiency for Graph Algorithm Execution".PROCEEDINGS OF THE VLDB ENDOWMENT 17.4(2023).