Neuro-Symbolic Speech Understanding in Aircraft Maintenance Metaverse

In the emerging world of metaverses, it is essential for speech communication systems to be aware of context to interact with virtual assets in the 3D world. This paper proposes the metaverse for aircraft maintenance training and education of Boeing-737, supplied with legacy manuals, 3D models, 3D s...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Aziz Siyaev, Geun-Sik Jo
Formato:	article
Lenguaje:	EN
Publicado:	IEEE 2021
Materias:	Aircraft maintenance education Boeing-737 deep learning industry 40 metaverse mixed reality Electrical engineering. Electronics. Nuclear engineering TK1-9971
Acceso en línea:	https://doaj.org/article/006400b4cfb94af1b825ae170cec6503
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:006400b4cfb94af1b825ae170cec6503
record_format	dspace
spelling	oai:doaj.org-article:006400b4cfb94af1b825ae170cec65032021-12-01T00:01:42ZNeuro-Symbolic Speech Understanding in Aircraft Maintenance Metaverse2169-353610.1109/ACCESS.2021.3128616https://doaj.org/article/006400b4cfb94af1b825ae170cec65032021-01-01T00:00:00Zhttps://ieeexplore.ieee.org/document/9617584/https://doaj.org/toc/2169-3536In the emerging world of metaverses, it is essential for speech communication systems to be aware of context to interact with virtual assets in the 3D world. This paper proposes the metaverse for aircraft maintenance training and education of Boeing-737, supplied with legacy manuals, 3D models, 3D simulators, and aircraft maintenance knowledge. Furthermore, to navigate and control operational flow in the metaverse, which is strictly followed by maintenance manuals, the context-aware speech understanding module Neuro-Symbolic Speech Executor (NSSE) is presented. Unlike conventional speech recognition methods, NSSE applies Neuro-Symbolic AI, which combines neural networks and traditional symbolic reasoning, to understand users’ requests and reply based on context and aircraft-specific knowledge. NSSE is developed with an industrially flexible approach by applying only synthetic data for training. Nevertheless, the evaluation process performed with various automatic speech recognition metrics on real users’ data showed sustainable results with an average accuracy of 94.7%, Word Error Rate (WER) of 7.5%, and the generalization ability to handle speech requests of users with the non-native pronunciation. The proposed Aircraft Maintenance Metaverse is a cheap and scalable solution for aviation colleges since it replaces expensive physical aircraft with virtual one that can be easily modified and updated. Moreover, the Neuro-Symbolic Speech Executor, playing the role of field expert, provides technical guidance and all the resources to facilitate effective training and education of aircraft maintenance.Aziz SiyaevGeun-Sik JoIEEEarticleAircraft maintenance educationBoeing-737deep learningindustry 40metaversemixed realityElectrical engineering. Electronics. Nuclear engineeringTK1-9971ENIEEE Access, Vol 9, Pp 154484-154499 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	Aircraft maintenance education Boeing-737 deep learning industry 40 metaverse mixed reality Electrical engineering. Electronics. Nuclear engineering TK1-9971
spellingShingle	Aircraft maintenance education Boeing-737 deep learning industry 40 metaverse mixed reality Electrical engineering. Electronics. Nuclear engineering TK1-9971 Aziz Siyaev Geun-Sik Jo Neuro-Symbolic Speech Understanding in Aircraft Maintenance Metaverse
description	In the emerging world of metaverses, it is essential for speech communication systems to be aware of context to interact with virtual assets in the 3D world. This paper proposes the metaverse for aircraft maintenance training and education of Boeing-737, supplied with legacy manuals, 3D models, 3D simulators, and aircraft maintenance knowledge. Furthermore, to navigate and control operational flow in the metaverse, which is strictly followed by maintenance manuals, the context-aware speech understanding module Neuro-Symbolic Speech Executor (NSSE) is presented. Unlike conventional speech recognition methods, NSSE applies Neuro-Symbolic AI, which combines neural networks and traditional symbolic reasoning, to understand users’ requests and reply based on context and aircraft-specific knowledge. NSSE is developed with an industrially flexible approach by applying only synthetic data for training. Nevertheless, the evaluation process performed with various automatic speech recognition metrics on real users’ data showed sustainable results with an average accuracy of 94.7%, Word Error Rate (WER) of 7.5%, and the generalization ability to handle speech requests of users with the non-native pronunciation. The proposed Aircraft Maintenance Metaverse is a cheap and scalable solution for aviation colleges since it replaces expensive physical aircraft with virtual one that can be easily modified and updated. Moreover, the Neuro-Symbolic Speech Executor, playing the role of field expert, provides technical guidance and all the resources to facilitate effective training and education of aircraft maintenance.
format	article
author	Aziz Siyaev Geun-Sik Jo
author_facet	Aziz Siyaev Geun-Sik Jo
author_sort	Aziz Siyaev
title	Neuro-Symbolic Speech Understanding in Aircraft Maintenance Metaverse
title_short	Neuro-Symbolic Speech Understanding in Aircraft Maintenance Metaverse
title_full	Neuro-Symbolic Speech Understanding in Aircraft Maintenance Metaverse
title_fullStr	Neuro-Symbolic Speech Understanding in Aircraft Maintenance Metaverse
title_full_unstemmed	Neuro-Symbolic Speech Understanding in Aircraft Maintenance Metaverse
title_sort	neuro-symbolic speech understanding in aircraft maintenance metaverse
publisher	IEEE
publishDate	2021
url	https://doaj.org/article/006400b4cfb94af1b825ae170cec6503
work_keys_str_mv	AT azizsiyaev neurosymbolicspeechunderstandinginaircraftmaintenancemetaverse AT geunsikjo neurosymbolicspeechunderstandinginaircraftmaintenancemetaverse
_version_	1718406120446361600

Neuro-Symbolic Speech Understanding in Aircraft Maintenance Metaverse

Ejemplares similares