结合主题感知与通信代理的文本摘要模型

doi:10.19665/j.issn1001-2400.2020.03.014

Abstract

Abstract:

To solve the problem that the traditional automatic text summary model cannot generate a high-quality long text summary due to the limitation of the length of the RNN (Recurrent Neural Network), a model of abstractive text summarization for topic-aware communicating agents has been proposed. First, the problem that the LSTM (Long Short-Term Memory) input sequence is too long to generate the abstract with prior information has been solved by dividing the encoder into multiple collaborating agents. Then for providing topic information and improving the correlation between the generated abstract and the source text, the joint attention mechanism has been added into our model. Finally, a hybrid training method with reinforcement learning has been employed in order to solve the problem of exposure bias, and optimize the evaluation index directly. The results show that our model not only generate long text summaries with prominent themes, but also has a higher score than the state-of-the-art models, which indicates that with the help of topic information, the model for communicating agents can be expected to generate long text summaries better.

Key words: abstractive text summarization, communicating agents, topic awareness, joint attention, reinforcement learning

CLC Number:

TP183

ZHANG Zheming,REN Shuxia,GUO Kaijie. Model of abstractive text summarization for topic-aware communicating agents[J].Journal of Xidian University, 2020, 47(3): 97-104.

Figures/Tables 3

References 15

[1]	NALLAPATI R, ZHAI F, ZHOU B. Summarunner: a Recurrent Neural Network Based Sequence Model for Extractive Summarization of Documents[C]//Proceedings of the 2017 31st AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2017: 3075-3081.
[2]	CELIKYILMAZ A, BOSSELUT A, HE X D, et al. Deep Communicating Agents for Abstractive Summarization[C/OL]. [2019-10-22].https://arxiv.org/pdf/1803.10357.pdf.
[3]	石磊, 阮选敏, 魏瑞斌, 等. 基于序列到序列模型的生成式文本摘要研究综述[J]. 情报学报, 2019,38(10):1102-1116.
	SHI Lei, RUAN Xuanmin, WEI Ruibin, et al. Abstractive Summarization Based on Sequence to Sequence Models: A Review[J]. Journal of the China Society for Scientific and Technical Information, 2019,38(10):1102-1116.
[4]	RUSH A M, CHOPRA S, WESTON J. A Neural Attention Model for Abstractive Sentence Summarization[C/OL]. [2019-10-22].https://arxiv.org/pdf/1509.00685.pdf.
[5]	SEE A, LIU P J, MANNING C D. Get to the Point: Summarization with Pointer-generator Networks[C/OL]. [2019-10-22].https://arxiv.org/pdf/1704.04368.pdf.
[6]	GEHRING J, AULI M, GRANGIER D, et al. Convolutional Sequence to Sequence Learning[C/OL]. [2019-10-22].https://arxiv.org/pdf/1705.03122.pdf.
[7]	李勇, 成红红, 梁新彦, 等. CNN图像标题生成[J]. 西安电子科技大学学报, 2019,46(2):152-157.
	LI Yong, CHENG Honghong, LIANG Xinyan, et al. CNN Image Caption Generation[J]. Journal of Xidian University, 2019,46(2):152-157.
[8]	XING C, WU W, WU Y, et al. Topic Aware Neural Response Generation[C]//Proceedings of the 2017 31st AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2017: 3351-3357.
[9]	马书磊, 张国宾, 焦阳, 等. 一种改进的全局注意机制图像描述方法[J]. 西安电子科技大学学报, 2019,46(2):17-22.
	MA Shulei, ZHANG Guobin, JIAO Yang, et al. Improved Method for Image Caption with Global Attention Mechanism[J]. Journal of Xidian University, 2019,46(2):17-22.
[10]	PAULUS R, XIONG C, SOCHER R. A Deep Reinforced Model for Abstractive Summarization[C/OL]. [2019-10-22].https://arxiv.org/pdf/1705.04304.pdf.
[11]	RENNIE S J, MARCHERET E, MROUEH Y, et al. Self-critical Sequence TRAINING for Image Captioning [C]//Proceedings of the 2017 30th IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 1179-1195.
[12]	HERMANN K M, KOCISKY T, GREFENSTETTE E, et al. Teaching Machines to Read and Comprehend [C]//Advances in Neural Information Processing Systems. Vancouver, Canada: Neural Information Processing Systems Foundation, 2015: 1693-1701.
[13]	LIN C Y, HOVY E. Automatic Evaluation of Summaries Using N-gram Co-occurrence Statistics [C]//Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology: 1. New York: Association for Computational Linguistics, 2003: 71-78.
[14]	ZHOU Q, YANG N, WEI F, et al. Neural Document Summarization by Jointly Learning to Score and Select Sentences[C/OL]. [2019-10-22].https://arxiv.org/pdf/1807.02305.pdf.
[15]	CHEN Y C, BANSAL M. Fast Abstractive Summarization with Reinforce-selected Sentence Rewriting [C]//Proceedings of the 2018 56th Annual Meeting of the Association for Computational Linguistics. New York: Association for Computational Linguistics, 2018: 675-686.

基线模型	模型描述
SummaRuNNer^[1]	一个简单的基于RNN的抽取式模型,使用的是端到端的训练机制。此模型在CNN/DailyMail数据集上的ROUGE评分是2017年最高的
pointer-generator与pointer-generator +coverage^[5]	一个新颖的指针生成网络模型,解决了未登录词(OOV)问题。pointer-generator+coverage在此基础上又加入了覆盖机制,解决了摘要生成时可能出现的自我重复问题,提高了生成摘要的质量
ML+RL, with intra-attention^[10]	一个基于RNN同时结合了内注意力机制的模型。这篇文章第一次将强化学习的训练方法用于自动文本摘要模型的训练之中。ROUGE评分较之前的模型有明显的提升
PGN^[14]	一个端到端的抽取式文本摘要模型,将选择策略集成到打分模型中,解决了句子打分和句子选择这两部分割裂的问题
rnn-ext+abs+RL+rerank^[15]	一个准确快速的摘要模型,提出了一种用于生成式摘要的句子级别的强化学习技术,在不需要标注关键句子语料的情况下,实现抽取式和生成式的结合
DCA MLE+SEM+RL, mpgen, with-comm, with caa (3-agents)^[2]	一个结合了深度通信代理,指针网络以及语义联合的模型,并加入了强化学习的混合训练方式

模型	ROUGE-1	ROUGE-2	ROUGE-L
SummaRuNNer^[1] pointer-generator^[5] pointer-generator + coverage^[5] ML, with intra-attention^[10] RL, with intra-attention^[10] ML + RL, with intra-attention^[10] PGN^[14] rnn-ext + abs + RL + rerank^[15]	39.60 36.44 39.53 38.30 41.16 39.87 39.53 39.66	16.20 15.66 17.28 14.81 15.75 15.82 17.28 15.85	35.30 33.42 36.38 35.49 39.08 36.90 36.38 37.34
DCA MLE + SEM + RL, mpgen, with-comm, with caa (3-agents)^[2]	41.69	19.47	37.92
TADCA + ML (our baseline-1) TADCA + RL (our baseline-2) TADCA + MIX (our model)	41.11 43.88 42.97	18.15 20.03 20.53	37.05 40.57 38.72

Model of abstractive text summarization for topic-aware communicating agents

RichHTML

PDF (PC)

Like

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 3

References 15

Related Articles 6

Metrics

Comments

Recommended 10

[1]	LV Wenkai,YANG Pengfei,DING Yunqing,ZHANG Heyu,ZHENG Tianyang. JEDERL:A task scheduling optimization algorithm for heterogeneous computing platforms [J]. Journal of Xidian University, 2021, 48(6): 67-74.
[2]	LI Teng,CAO Shijie,YIN Siwei,WEI Dawei,MA Xindi,MA Jianfeng. Optimal method for the generation of the attack path based on the Q-learning decision [J]. Journal of Xidian University, 2021, 48(1): 160-167.
[3]	ZHANG Ying,WEI Minfeng,WANG Shihui,TAO Leiyan,CAO Jian,ZHANG Xing. Aircraft reinforcement learning multi-mode control in orbit [J]. Journal of Xidian University, 2020, 47(2): 75-82.
[4]	ZHOU Yi,CHEN Bo. Method for robot obstacle avoidance based on the improved dueling network [J]. Journal of Xidian University, 2019, 46(1): 46-50.
[5]	ZHANG Wenzhu;SHAO Lina. Dynamic spectrum allocation algorithm for heterogeneous radio networks based on reinforcement learning [J]. J4, 2011, 38(4): 32-37.
[6]	ZHANG Xin1;2;CHANG Yi-lin1;SHEN Zhong1;LIU Fu-jie1;CUI Can1. Fault monitoring policy for a hierarchical network with a multi-manager [J]. J4, 2005, 32(6): 873-876.