ORGAN: Observation-Guided Radiology Report Generation via Tree Reasoning

Hou, Wenjun1,2; Xu, Kaishuai1; Cheng, Yi1; Li, Wenjie1; Liu, Jiang2

2023

61st Annual Meeting of the the Association-for-Computational-Linguistics (ACL)

PROCEEDINGS OF THE 61ST ANNUAL MEETING OF THE ASSOCIATION FOR COMPUTATIONAL LINGUISTICS, ACL 2023, VOL 1

JUL 09-14, 2023

null,Toronto,CANADA

209 N EIGHTH STREET, STROUDSBURG, PA 18360 USA

ASSOC COMPUTATIONAL LINGUISTICS-ACL

This paper explores the task of radiology report generation, which aims at generating free-text descriptions for a set of radiographs. One significant challenge of this task is how to correctly maintain the consistency between the images and the lengthy report. Previous research explored solving this issue through planningbased methods, which generate reports only based on high-level plans. However, these plans usually only contain the major observations from the radiographs (e.g., lung opacity), lacking much necessary information, such as the observation characteristics and preliminary clinical diagnoses. To address this problem, the system should also take the image information into account together with the textual plan and perform stronger reasoning during the generation process. In this paper, we propose an Observation-guided radiology Report GenerAtioN framework (ORGAN). It first produces an observation plan and then feeds both the plan and radiographs for report generation, where an observation graph and a tree reasoning mechanism are adopted to precisely enrich the plan information by capturing the multiformats of each observation. Experimental results demonstrate that our framework outperforms previous state-of-the-art methods regarding text quality and clinical efficacy.(1)

通讯

英语

EI ; CPCI-S

General Program of National Natural Science Foundation of China["82272086","62076212"] ; Guangdong Provincial Department of Education[2020ZDZX3043] ; Shenzhen Natural Science Fund[JCYJ20200109140820699] ; Research Grants Council of Hong Kong["15207920","15207821","15207122"] ; Shenzhen Natural Science Fund (Stable Support Plan Program)[20200925174052004]

Computer Science

Computer Science, Artificial Intelligence ; Computer Science, Information Systems ; Computer Science, Theory & Methods

WOS:001181086807022

Web of Science

1.Department of Computing, The Hong Kong Polytechnic University, Hong Kong
2.Research Institute of Trustworthy Autonomous Systems, Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, China

Hou, Wenjun,Xu, Kaishuai,Cheng, Yi,et al. ORGAN: Observation-Guided Radiology Report Generation via Tree Reasoning[C]. 209 N EIGHTH STREET, STROUDSBURG, PA 18360 USA:ASSOC COMPUTATIONAL LINGUISTICS-ACL,2023.