Hybrid attention-based transformer block model for distant supervision relation extraction

Xiao，Yan1; Jin，Yaochu1,2,3; Cheng，Ran4; Hao，Kuangrong1

2022-01-22

10.1016/j.neucom.2021.10.037

NEUROCOMPUTING   影响因子和分区

0925-2312

1872-8286

With an exponential explosive growth of various digital text information, it is challenging to efficiently obtain specific knowledge from massive unstructured text information. As one basic task for natural language processing (NLP), relation extraction (RE) aims to extract semantic relations between entity pairs based on the given text. To avoid manual labeling of datasets, distant supervision relation extraction (DSRE) has been widely used, aiming to utilize knowledge base to automatically annotate datasets. Unfortunately, this method heavily suffers from wrong labelling due to its underlying strong assumptions. To address this issue, we propose a new framework using hybrid attention-based Transformer block with multi-instance learning for DSRE. More specifically, the Transformer block is, for the first time, used as a sentence encoder, which mainly utilizes multi-head self-attention to capture syntactic information at the word level. Then, a novel sentence-level attention mechanism is proposed to calculate the bag representation, aiming to exploit all useful information in each sentence. Experimental results on the public dataset New York Times (NYT) demonstrate that the proposed approach can outperform the state-of-the-art algorithms on the adopted dataset, which verifies the effectiveness of our model on the DSRE task.

Distant supervision relation extraction (DSRE) Sentence-level attention Transformer block

SCI ; EI

英语

其他

Fundamental Research Funds for the Central Universities["2232021A-10","2232021D-37"] ; National Natural Science Foundation of China[61806051] ; Natural Science Foundation of Shanghai["20ZR1400400","21ZR1401700"] ; Graduate Student Innovation Fund of Donghua University[CUSFDH-D-2021051]

Computer Science

Computer Science, Artificial Intelligence

WOS:000722305600003

ELSEVIER

20214611156163

Extraction ; Knowledge based systems ; Semantics

Data Processing and Image Processing:723.2 ; Expert Systems:723.4.1 ; Chemical Operations:802.3

COMPUTER SCIENCE

2-s2.0-85118890283

Scopus

1.Engineering Research Center of Digitized Textile & Apparel Technology,Ministry of Education,Donghua University,Shanghai,201620,China
2.Chair of Nature Inspired Computing and Engineering,Faculty of Technology,Bielefeld University,Bielefeld,D-33615,Germany
3.Department of Computer Science,University of Surrey,Guildford,GU2 7XH,United Kingdom
4.The Shenzhen Key Laboratory of Computational Intelligence,University Key Laboratory of Evolving Intelligent Systems of Guangdong Province,Department of Computer Science and Engineering,Southern University of Science and Technology,Shenzhen,518055,China

Xiao，Yan,Jin，Yaochu,Cheng，Ran,et al. Hybrid attention-based transformer block model for distant supervision relation extraction[J]. NEUROCOMPUTING,2022,470:29-39.

Xiao，Yan,Jin，Yaochu,Cheng，Ran,&Hao，Kuangrong.(2022).Hybrid attention-based transformer block model for distant supervision relation extraction.NEUROCOMPUTING,470,29-39.

Xiao，Yan,et al."Hybrid attention-based transformer block model for distant supervision relation extraction".NEUROCOMPUTING 470(2022):29-39.