Bi-Directional Recurrent Attentional Topic Model

Li，Shuangyin1; Zhang，Yu2; Pan，Rong3

2020-10-01

10.1145/3412371

ACM Transactions on Knowledge Discovery from Data   影响因子和分区

1556-4681

1556-472X

In a document, the topic distribution of a sentence depends on both the topics of its neighbored sentences and its own content, and it is usually affected by the topics of the neighbored sentences with different weights. The neighbored sentences of a sentence include the preceding sentences and the subsequent sentences. Meanwhile, it is natural that a document can be treated as a sequence of sentences. Most existing works for Bayesian document modeling do not take these points into consideration. To fill this gap, we propose a bi-Directional Recurrent Attentional Topic Model (bi-RATM) for document embedding. The bi-RATM not only takes advantage of the sequential orders among sentences but also uses the attention mechanism to model the relations among successive sentences. To support to the bi-RATM, we propose a bi-Directional Recurrent Attentional Bayesian Process (bi-RABP) to handle the sequences. Based on the bi-RABP, bi-RATM fully utilizes the bi-directional sequential information of the sentences in a document. Online bi-RATM is proposed to handle large-scale corpus. Experiments on two corpora show that the proposed model outperforms state-of-the-art methods on document modeling and classification.

Bi-directional recurrent attentions recurrent attentional Bayesian process topic modeling

SCI ; EI

英语

其他

Key-Area Research and Development Program of Guangdong Province[2019B010137003] ; National Key-Area Research and Development Program of China[2018YFB1404402] ; Basic and Applied Basic Research Fund of Guangdong Province[2019B1515120085] ; Guangdong Science and Technology Fund[2018A07071702]

Computer Science

Computer Science, Information Systems ; Computer Science, Software Engineering

WOS:000580421600010

ASSOC COMPUTING MACHINERY

20204309372371

Information Retrieval and Use:903.3

2-s2.0-85092726562

Scopus

1.School of Computer Science,South China Normal University,Key Lab on Cloud Security and Assessment Technology of Guangzhou,Tianhe District, Guangzhou, Guangdong,No. 55, West of Zhongshan Avenue,510631,China
2.Department of Computer Science and Engineering,Southern University of Science and Technology,Shenzhen, Guangdong,1088 Xueyuan Avenue,518055,China
3.Department of Data and Computer Science,Sun Yat-sen University,Guangzhou, Guangdong,Waihuan East Road 132,510006,China

Li，Shuangyin,Zhang，Yu,Pan，Rong. Bi-Directional Recurrent Attentional Topic Model[J]. ACM Transactions on Knowledge Discovery from Data,2020,14(6).

Li，Shuangyin,Zhang，Yu,&Pan，Rong.(2020).Bi-Directional Recurrent Attentional Topic Model.ACM Transactions on Knowledge Discovery from Data,14(6).

Li，Shuangyin,et al."Bi-Directional Recurrent Attentional Topic Model".ACM Transactions on Knowledge Discovery from Data 14.6(2020).