Sentence Compression Using BERT and Graph Convolutional Networks

Sentence compression is a natural language-processing task that produces a short paraphrase of an input sentence by deleting words from the input sentence while ensuring grammatical correctness and preserving meaningful core information. This study introduces a graph convolutional network (GCN) into...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Yo-Han Park, Gyong-Ho Lee, Yong-Seok Choi, Kong-Joo Lee
Formato:	article
Lenguaje:	EN
Publicado:	MDPI AG 2021
Materias:	dependency tree graph convolutional network graph neural networks pre-trained model sentence compression Technology T Engineering (General). Civil engineering (General) TA1-2040 Biology (General) QH301-705.5 Physics QC1-999 Chemistry QD1-999
Acceso en línea:	https://doaj.org/article/f662c617032640ec80e68ff4dde1c624
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:f662c617032640ec80e68ff4dde1c624
record_format	dspace
spelling	oai:doaj.org-article:f662c617032640ec80e68ff4dde1c6242021-11-11T15:00:37ZSentence Compression Using BERT and Graph Convolutional Networks10.3390/app112199102076-3417https://doaj.org/article/f662c617032640ec80e68ff4dde1c6242021-10-01T00:00:00Zhttps://www.mdpi.com/2076-3417/11/21/9910https://doaj.org/toc/2076-3417Sentence compression is a natural language-processing task that produces a short paraphrase of an input sentence by deleting words from the input sentence while ensuring grammatical correctness and preserving meaningful core information. This study introduces a graph convolutional network (GCN) into a sentence compression task to encode syntactic information, such as dependency trees. As we upgrade the GCN to activate a directed edge, the compression model with the GCN layers can distinguish between parent and child nodes in a dependency tree when aggregating adjacent nodes. Furthermore, by increasing the number of GCN layers, the model can gradually collect high-order information of a dependency tree when propagating node information through the layers. We implement a sentence compression model for Korean and English, respectively. This model consists of three components: pre-trained BERT model, GCN layers, and a scoring layer. The scoring layer can determine whether a word should remain in a compressed sentence by relying on the word vector containing contextual and syntactic information encoded by BERT and GCN layers. To train and evaluate the proposed model, we used the Google sentence compression dataset for English and a Korean sentence compression corpus containing about 140,000 sentence pairs for Korean. The experimental results demonstrate that the proposed model achieves state-of-the-art performance for English. To the best of our knowledge, this sentence compression model based on the deep learning model trained with a large-scale corpus is the first attempt for Korean.Yo-Han ParkGyong-Ho LeeYong-Seok ChoiKong-Joo LeeMDPI AGarticledependency treegraph convolutional networkgraph neural networkspre-trained modelsentence compressionTechnologyTEngineering (General). Civil engineering (General)TA1-2040Biology (General)QH301-705.5PhysicsQC1-999ChemistryQD1-999ENApplied Sciences, Vol 11, Iss 9910, p 9910 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	dependency tree graph convolutional network graph neural networks pre-trained model sentence compression Technology T Engineering (General). Civil engineering (General) TA1-2040 Biology (General) QH301-705.5 Physics QC1-999 Chemistry QD1-999
spellingShingle	dependency tree graph convolutional network graph neural networks pre-trained model sentence compression Technology T Engineering (General). Civil engineering (General) TA1-2040 Biology (General) QH301-705.5 Physics QC1-999 Chemistry QD1-999 Yo-Han Park Gyong-Ho Lee Yong-Seok Choi Kong-Joo Lee Sentence Compression Using BERT and Graph Convolutional Networks
description	Sentence compression is a natural language-processing task that produces a short paraphrase of an input sentence by deleting words from the input sentence while ensuring grammatical correctness and preserving meaningful core information. This study introduces a graph convolutional network (GCN) into a sentence compression task to encode syntactic information, such as dependency trees. As we upgrade the GCN to activate a directed edge, the compression model with the GCN layers can distinguish between parent and child nodes in a dependency tree when aggregating adjacent nodes. Furthermore, by increasing the number of GCN layers, the model can gradually collect high-order information of a dependency tree when propagating node information through the layers. We implement a sentence compression model for Korean and English, respectively. This model consists of three components: pre-trained BERT model, GCN layers, and a scoring layer. The scoring layer can determine whether a word should remain in a compressed sentence by relying on the word vector containing contextual and syntactic information encoded by BERT and GCN layers. To train and evaluate the proposed model, we used the Google sentence compression dataset for English and a Korean sentence compression corpus containing about 140,000 sentence pairs for Korean. The experimental results demonstrate that the proposed model achieves state-of-the-art performance for English. To the best of our knowledge, this sentence compression model based on the deep learning model trained with a large-scale corpus is the first attempt for Korean.
format	article
author	Yo-Han Park Gyong-Ho Lee Yong-Seok Choi Kong-Joo Lee
author_facet	Yo-Han Park Gyong-Ho Lee Yong-Seok Choi Kong-Joo Lee
author_sort	Yo-Han Park
title	Sentence Compression Using BERT and Graph Convolutional Networks
title_short	Sentence Compression Using BERT and Graph Convolutional Networks
title_full	Sentence Compression Using BERT and Graph Convolutional Networks
title_fullStr	Sentence Compression Using BERT and Graph Convolutional Networks
title_full_unstemmed	Sentence Compression Using BERT and Graph Convolutional Networks
title_sort	sentence compression using bert and graph convolutional networks
publisher	MDPI AG
publishDate	2021
url	https://doaj.org/article/f662c617032640ec80e68ff4dde1c624
work_keys_str_mv	AT yohanpark sentencecompressionusingbertandgraphconvolutionalnetworks AT gyongholee sentencecompressionusingbertandgraphconvolutionalnetworks AT yongseokchoi sentencecompressionusingbertandgraphconvolutionalnetworks AT kongjoolee sentencecompressionusingbertandgraphconvolutionalnetworks
_version_	1718437944738447360

Sentence Compression Using BERT and Graph Convolutional Networks

Ejemplares similares