Unified Transformer Multi-Task Learning for Intent Classification With Entity Recognition

Intent classification (IC) and Named Entity Recognition (NER) are arguably the two main components needed to build a Natural Language Understanding (NLU) engine, which is a main component of conversational agents. The IC and NER components are closely intertwined and the entities are often connected...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Alberto Benayas, Reyhaneh Hashempour, Damian Rumble, Shoaib Jameel, Renato Cordeiro De Amorim
Formato:	article
Lenguaje:	EN
Publicado:	IEEE 2021
Materias:	Intent classification named entity recognition multi-task learning transfer learning Electrical engineering. Electronics. Nuclear engineering TK1-9971
Acceso en línea:	https://doaj.org/article/367a5f976a734b5e8f1d82fe5e13a322
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:367a5f976a734b5e8f1d82fe5e13a322
record_format	dspace
spelling	oai:doaj.org-article:367a5f976a734b5e8f1d82fe5e13a3222021-11-18T00:10:45ZUnified Transformer Multi-Task Learning for Intent Classification With Entity Recognition2169-353610.1109/ACCESS.2021.3124268https://doaj.org/article/367a5f976a734b5e8f1d82fe5e13a3222021-01-01T00:00:00Zhttps://ieeexplore.ieee.org/document/9599152/https://doaj.org/toc/2169-3536Intent classification (IC) and Named Entity Recognition (NER) are arguably the two main components needed to build a Natural Language Understanding (NLU) engine, which is a main component of conversational agents. The IC and NER components are closely intertwined and the entities are often connected to the underlying intent. Current research has primarily focused to model IC and NER as two separate units, which results in error propagation, and thus, sub-optimal performance. In this paper, we propose a simple yet effective novel framework for NLU where the parameters of the IC and the NER models are jointly trained in a consolidated parameter space. Text semantic representations are obtained from popular pre-trained contextual language models, which are fine-tuned for our task, and these parameters are propagated to other deep neural layers in our framework leading to a faithful unified modelling of the IC and NER parameters. The overall framework results in a faithful parameter sharing when the training is underway, leading to a more coherent learning. Experiments on two public datasets, ATIS and SNIPS, show that our model outperforms other methods by a noticeable margin. On the SNIPS dataset, we obtain a 1.42% improvement in NER in terms of the F1 score, and 1% improvement in intent accuracy score. On ATIS, we achieve 1.54% improvement in intent accuracy score. We also present qualitative results to showcase the effectiveness of our model.Alberto BenayasReyhaneh HashempourDamian RumbleShoaib JameelRenato Cordeiro De AmorimIEEEarticleIntent classificationnamed entity recognitionmulti-task learningtransfer learningElectrical engineering. Electronics. Nuclear engineeringTK1-9971ENIEEE Access, Vol 9, Pp 147306-147314 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	Intent classification named entity recognition multi-task learning transfer learning Electrical engineering. Electronics. Nuclear engineering TK1-9971
spellingShingle	Intent classification named entity recognition multi-task learning transfer learning Electrical engineering. Electronics. Nuclear engineering TK1-9971 Alberto Benayas Reyhaneh Hashempour Damian Rumble Shoaib Jameel Renato Cordeiro De Amorim Unified Transformer Multi-Task Learning for Intent Classification With Entity Recognition
description	Intent classification (IC) and Named Entity Recognition (NER) are arguably the two main components needed to build a Natural Language Understanding (NLU) engine, which is a main component of conversational agents. The IC and NER components are closely intertwined and the entities are often connected to the underlying intent. Current research has primarily focused to model IC and NER as two separate units, which results in error propagation, and thus, sub-optimal performance. In this paper, we propose a simple yet effective novel framework for NLU where the parameters of the IC and the NER models are jointly trained in a consolidated parameter space. Text semantic representations are obtained from popular pre-trained contextual language models, which are fine-tuned for our task, and these parameters are propagated to other deep neural layers in our framework leading to a faithful unified modelling of the IC and NER parameters. The overall framework results in a faithful parameter sharing when the training is underway, leading to a more coherent learning. Experiments on two public datasets, ATIS and SNIPS, show that our model outperforms other methods by a noticeable margin. On the SNIPS dataset, we obtain a 1.42% improvement in NER in terms of the F1 score, and 1% improvement in intent accuracy score. On ATIS, we achieve 1.54% improvement in intent accuracy score. We also present qualitative results to showcase the effectiveness of our model.
format	article
author	Alberto Benayas Reyhaneh Hashempour Damian Rumble Shoaib Jameel Renato Cordeiro De Amorim
author_facet	Alberto Benayas Reyhaneh Hashempour Damian Rumble Shoaib Jameel Renato Cordeiro De Amorim
author_sort	Alberto Benayas
title	Unified Transformer Multi-Task Learning for Intent Classification With Entity Recognition
title_short	Unified Transformer Multi-Task Learning for Intent Classification With Entity Recognition
title_full	Unified Transformer Multi-Task Learning for Intent Classification With Entity Recognition
title_fullStr	Unified Transformer Multi-Task Learning for Intent Classification With Entity Recognition
title_full_unstemmed	Unified Transformer Multi-Task Learning for Intent Classification With Entity Recognition
title_sort	unified transformer multi-task learning for intent classification with entity recognition
publisher	IEEE
publishDate	2021
url	https://doaj.org/article/367a5f976a734b5e8f1d82fe5e13a322
work_keys_str_mv	AT albertobenayas unifiedtransformermultitasklearningforintentclassificationwithentityrecognition AT reyhanehhashempour unifiedtransformermultitasklearningforintentclassificationwithentityrecognition AT damianrumble unifiedtransformermultitasklearningforintentclassificationwithentityrecognition AT shoaibjameel unifiedtransformermultitasklearningforintentclassificationwithentityrecognition AT renatocordeirodeamorim unifiedtransformermultitasklearningforintentclassificationwithentityrecognition
_version_	1718425154424406016

Unified Transformer Multi-Task Learning for Intent Classification With Entity Recognition

Ejemplares similares