Biomedical Text Similarity Evaluation Using Attention Mechanism and Siamese Neural Network

It is a crucial component to estimate the similarity of biomedical sentence pair. Siamese neural network (SNN) can achieve better performance for non-biomedical corpora. However, SNN alone cannot obtain satisfactory biomedical text similarity evaluation results due to syntactic complexity and long s...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Zhengguang Li, Heng Chen, Huayue Chen
Formato:	article
Lenguaje:	EN
Publicado:	IEEE 2021
Materias:	Self-attention cross attention siamese network semantic textual similarity interactive semantic information Electrical engineering. Electronics. Nuclear engineering TK1-9971
Acceso en línea:	https://doaj.org/article/e443a926999d4d91a49be68a09fccc5a
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:e443a926999d4d91a49be68a09fccc5a
record_format	dspace
spelling	oai:doaj.org-article:e443a926999d4d91a49be68a09fccc5a2021-11-19T00:06:55ZBiomedical Text Similarity Evaluation Using Attention Mechanism and Siamese Neural Network2169-353610.1109/ACCESS.2021.3099021https://doaj.org/article/e443a926999d4d91a49be68a09fccc5a2021-01-01T00:00:00Zhttps://ieeexplore.ieee.org/document/9492136/https://doaj.org/toc/2169-3536It is a crucial component to estimate the similarity of biomedical sentence pair. Siamese neural network (SNN) can achieve better performance for non-biomedical corpora. However, SNN alone cannot obtain satisfactory biomedical text similarity evaluation results due to syntactic complexity and long sentences. In this paper, a cross self-attention (CSA) is proposed to design a new attention mechanism, namely self2self-attention(S2SA). Then the S2SA is introduced into SNN to construct a novel self2self-attentive siamese neural network, namely S2SA-SNN. In the S2SA-SNN, self-attention is used to learn the different weights of words and complex syntactic features in a single sentence. The means of the CSA are used to learn inherent interactive semantic information between sentences, and it employs self-attention instead of global attention to perform cross attention between sentences. Finally, three biomedical benchmark datasets of Pearson Correlation of 0.66 and 0.72/0.66 on DBMI and CDD-ful/-ref are used to test and prove the effectiveness of the S2SA-SNN. The experiment results show that the S2SA-SNN can achieve better performances with pre-trained word embedding and obtain better generalization ability than other compared methods.Zhengguang LiHeng ChenHuayue ChenIEEEarticleSelf-attentioncross attentionsiamese networksemantic textual similarityinteractive semantic informationElectrical engineering. Electronics. Nuclear engineeringTK1-9971ENIEEE Access, Vol 9, Pp 105002-105011 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	Self-attention cross attention siamese network semantic textual similarity interactive semantic information Electrical engineering. Electronics. Nuclear engineering TK1-9971
spellingShingle	Self-attention cross attention siamese network semantic textual similarity interactive semantic information Electrical engineering. Electronics. Nuclear engineering TK1-9971 Zhengguang Li Heng Chen Huayue Chen Biomedical Text Similarity Evaluation Using Attention Mechanism and Siamese Neural Network
description	It is a crucial component to estimate the similarity of biomedical sentence pair. Siamese neural network (SNN) can achieve better performance for non-biomedical corpora. However, SNN alone cannot obtain satisfactory biomedical text similarity evaluation results due to syntactic complexity and long sentences. In this paper, a cross self-attention (CSA) is proposed to design a new attention mechanism, namely self2self-attention(S2SA). Then the S2SA is introduced into SNN to construct a novel self2self-attentive siamese neural network, namely S2SA-SNN. In the S2SA-SNN, self-attention is used to learn the different weights of words and complex syntactic features in a single sentence. The means of the CSA are used to learn inherent interactive semantic information between sentences, and it employs self-attention instead of global attention to perform cross attention between sentences. Finally, three biomedical benchmark datasets of Pearson Correlation of 0.66 and 0.72/0.66 on DBMI and CDD-ful/-ref are used to test and prove the effectiveness of the S2SA-SNN. The experiment results show that the S2SA-SNN can achieve better performances with pre-trained word embedding and obtain better generalization ability than other compared methods.
format	article
author	Zhengguang Li Heng Chen Huayue Chen
author_facet	Zhengguang Li Heng Chen Huayue Chen
author_sort	Zhengguang Li
title	Biomedical Text Similarity Evaluation Using Attention Mechanism and Siamese Neural Network
title_short	Biomedical Text Similarity Evaluation Using Attention Mechanism and Siamese Neural Network
title_full	Biomedical Text Similarity Evaluation Using Attention Mechanism and Siamese Neural Network
title_fullStr	Biomedical Text Similarity Evaluation Using Attention Mechanism and Siamese Neural Network
title_full_unstemmed	Biomedical Text Similarity Evaluation Using Attention Mechanism and Siamese Neural Network
title_sort	biomedical text similarity evaluation using attention mechanism and siamese neural network
publisher	IEEE
publishDate	2021
url	https://doaj.org/article/e443a926999d4d91a49be68a09fccc5a
work_keys_str_mv	AT zhengguangli biomedicaltextsimilarityevaluationusingattentionmechanismandsiameseneuralnetwork AT hengchen biomedicaltextsimilarityevaluationusingattentionmechanismandsiameseneuralnetwork AT huayuechen biomedicaltextsimilarityevaluationusingattentionmechanismandsiameseneuralnetwork
_version_	1718420641823064064

Biomedical Text Similarity Evaluation Using Attention Mechanism and Siamese Neural Network

Ejemplares similares