基于語義理解注意力神經(jīng)網(wǎng)絡(luò)的多元特征融合中文文本分類
doi: 10.11999/JEIT170815
基金項(xiàng)目:
黑龍江省海外學(xué)人基金(1253HQ019)
Multi-feature Fusion Based on Semantic Understanding Attention Neural Network for Chinese Text Categorization
Funds:
The Overseas Scholars Fund Project of Heilongjiang Province (1253HQ019)
-
摘要: 在中文文本分類任務(wù)中，針對重要特征在中文文本中位置分布分散、稀疏的問題，以及不同文本特征對文本類別識別貢獻(xiàn)不同的問題，該文提出一種基于語義理解的注意力神經(jīng)網(wǎng)絡(luò)、長短期記憶網(wǎng)絡(luò)(LSTM)與卷積神經(jīng)網(wǎng)絡(luò)(CNN)的多元特征融合中文文本分類模型(3CLA)。模型首先通過文本預(yù)處理將中文文本分詞、向量化。然后，通過嵌入層分別經(jīng)過CNN通路、LSTM通路和注意力算法模型通路以提取不同層次、具有不同特點(diǎn)的文本特征。最終，文本特征經(jīng)融合層融合后，由softmax分類器進(jìn)行分類?；谥形恼Z料進(jìn)行了文本分類實(shí)驗(yàn)。實(shí)驗(yàn)結(jié)果表明，相較于CNN結(jié)構(gòu)模型與LSTM結(jié)構(gòu)模型，提出的算法模型對中文文本類別的識別能力最多提升約8%。
-
關(guān)鍵詞:
- 中文文本分類 /
- 多元特征融合 /
- 注意力算法 /
- 長短期記憶網(wǎng)絡(luò) /
- 卷積神經(jīng)網(wǎng)絡(luò)
Abstract: In Chinese text categorization tasks, the locations of the important features in the Chinese texts are disperse and sparse, and the different characteristics of Chinese texts contributes differently for the recognition of their categories. In order to solve the above problems, this paper proposes a multi-feature fusion model Three Convolutional neural network paths and Long short term memory path fused with Attention neural network path (3CLA) for Chinese text categorization, which is based on Convolutional Neural Network (CNN), Long Short Term Memory (LSTM) and semantic understanding attention neural networks. The model first uses text preprocessing to finish the segmentation and vectorization of the Chinese text. Then, through the embedding layer, the input data are sent to the CNN path, the LSTM path and the attention path respectively to extract text features of different levels and different characteristics. Finally, the text features are fused by the fusion layer and classified by the classifier. Based on the Chinese corpus, the text classification experiment is carried out. The results of the experiments show that compared with the CNN structure model and the LSTM structure model, the proposed algorithm model improves the recognition ability of Chinese text categories by up to about 8%. -
孫曉, 彭曉琪, 胡敏, 等. 基于多維擴(kuò)展特征與深度學(xué)習(xí)的微博短文本情感分析[J]. 電子與信息學(xué)報(bào), 2017, 39(9): 2048-2055. doi: 10.11999/JEIT160975. SUN Xiao, PENG Xiaoqi, HU Min, et al. Extended multi- modality features and deep learning based Microblog short text sentiment analysis[J]. Journal of Electronics Information Technology, 2017, 39(9): 2048-2055. doi: 10.11999/JEIT160975. KIM Yoon. Convolutional neural networks for sentence classification[C]. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar, 2014: 1746-1751. PHAM Ngocquan, KRUSZEWSKI German, and BOLEDA Gemma. Convolutional neural network language models[C]. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin Texas, 2016: 1153-1162. 陳釗, 徐睿峰, 桂林, 等. 結(jié)合卷積神經(jīng)網(wǎng)絡(luò)和詞語情感序列特征的中文情感分析[J]. 中文信息學(xué)報(bào), 2015, 29(6): 172-178. doi: 10.3969/j.issn.1003-0077.2015.06.023. CHEN Zhao, XU Ruifeng, GUI Lin, et al. Combining convolutional neural networks and word sentiment sequence features for Chinese text sentiment analysis[J]. Journal of Chinese Information Processing, 2015, 29(6): 172-178. doi: 10.3969/j.issn.1003-0077.2015.06.023. 劉龍飛, 楊亮, 張紹武, 等. 基于卷積神經(jīng)網(wǎng)絡(luò)的微博情感傾向性分析[J]. 中文信息學(xué)報(bào), 2015, 29(6): 159-165. doi: 10.3969/j.issn.1003-0077.2015.06.021. LIU Longfei, YANG Liang, ZHANG Shaowu, et al. Convolutional neural networks for Chinese micro-blog sentiment analysis[J]. Journal of Chinese Information Processing, 2015, 29(6): 159-165. doi: 10.3969/j.issn.1003- 0077.2015.06.021. ZHANG Ye, MARSHALL Iain, and WALLACE B C. Rational-augmented convolutional neural networks for text classification[C]. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin Texas, 2016: 759-804. MIYAMOTO Yasumasa and CHO Kyunghyun. Gated word-character recurrent language model[C]. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin Texas, 2016: 1992-1997. TANG Duyu, QIN Bing, LIU Ting, et al. Document modeling with gated recurrent neural network for sentiment classification[C]. Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Austin Texas, 2015: 1422-1432. 梁軍, 柴玉梅, 原慧斌, 等. 基于極性轉(zhuǎn)移和LSTM遞歸網(wǎng)絡(luò)的情感分析[J]. 中文信息學(xué)報(bào), 2015, 29(5): 160-167. doi: 10.3969/j.issn.1003-0077.2015.05.020. LIANG Jun, CHAI Yumei, YUAN Huibin, et al. Polarity shifting and LSTM based recursive networks for sentiment analysis[J]. Journal of Chinese Information Processing, 2015, 29(5): 160-167. doi: 10.3969/j.issn.1003-0077.2015.05.020. 劉飛龍, 郝文寧, 陳剛, 等. 基于雙線性函數(shù)注意力Bi-LSTM模型的機(jī)器閱讀理解[J]. 計(jì)算機(jī)科學(xué), 2017, 44(6A): 92-96. doi: 10.11896/j.issn.1002-137X.2017.6A.019. LIU Feilong, HAO Wenning, CHEN Gang, et al. Attention of bilinear function based Bi-LSTM model for machine reading comprehension[J]. Computer Science, 2017, 44(6A): 92-96. doi: 10.11896/j.issn.1002-137X.2017.6A.019. CHENG Jianpeng, DONG Li, and LAPATA Mirella. Long short-term memory-networks for machine reading[C]. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin Texas, 2016: 551-561. WANG Yequan, HUANG Minlie, ZHAO Li, et al. Attention- based LSTM for aspect-level sentiment classification[C]. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin Texas, 2016: 606-615. PARIKH Ankur P, TAKSTROM Oscar, DAS Dipanjan, et al. A decomposable attention model for natural language inference[C]. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin Texas, 2016: 2249-2255. GOLUB David and HE Xiaodong. Character-level question answering with attention[C]. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin Texas, 2016: 1598-1607. MI Haitao, WANG Zhiguo, and ITTYCHERIAH Abe. Supervised attentions for neural machine translation[C]. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin Texas, 2016: 2283-2288. -
計(jì)量
- 文章訪問數(shù): 2016
- HTML全文瀏覽量: 355
- PDF下載量: 341
- 被引次數(shù): 0
下載:
