The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence

Abstract Real-world experience is typically multimodal. Evidence indicates that the facilitation in the detection of multisensory stimuli is modulated by the perceptual load, the amount of information involved in the processing of the stimuli. Here, we used a realistic virtual reality environment wh...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Matteo Marucci, Gianluca Di Flumeri, Gianluca Borghini, Nicolina Sciaraffa, Michele Scandola, Enea Francesco Pavone, Fabio Babiloni, Viviana Betti, Pietro Aricò
Formato:	article
Lenguaje:	EN
Publicado:	Nature Portfolio 2021
Materias:	Medicine R Science Q
Acceso en línea:	https://doaj.org/article/2e1e1e55954b425c8ef20a09f25dfcab
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:2e1e1e55954b425c8ef20a09f25dfcab
record_format	dspace
spelling	oai:doaj.org-article:2e1e1e55954b425c8ef20a09f25dfcab2021-12-02T15:54:13ZThe impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence10.1038/s41598-021-84196-82045-2322https://doaj.org/article/2e1e1e55954b425c8ef20a09f25dfcab2021-03-01T00:00:00Zhttps://doi.org/10.1038/s41598-021-84196-8https://doaj.org/toc/2045-2322Abstract Real-world experience is typically multimodal. Evidence indicates that the facilitation in the detection of multisensory stimuli is modulated by the perceptual load, the amount of information involved in the processing of the stimuli. Here, we used a realistic virtual reality environment while concomitantly acquiring Electroencephalography (EEG) and Galvanic Skin Response (GSR) to investigate how multisensory signals impact target detection in two conditions, high and low perceptual load. Different multimodal stimuli (auditory and vibrotactile) were presented, alone or in combination with the visual target. Results showed that only in the high load condition, multisensory stimuli significantly improve performance, compared to visual stimulation alone. Multisensory stimulation also decreases the EEG-based workload. Instead, the perceived workload, according to the “NASA Task Load Index” questionnaire, was reduced only by the trimodal condition (i.e., visual, auditory, tactile). This trimodal stimulation was more effective in enhancing the sense of presence, that is the feeling of being in the virtual environment, compared to the bimodal or unimodal stimulation. Also, we show that in the high load task, the GSR components are higher compared to the low load condition. Finally, the multimodal stimulation (Visual-Audio-Tactile—VAT and Visual-Audio—VA) induced a significant decrease in latency, and a significant increase in the amplitude of the P300 potentials with respect to the unimodal (visual) and visual and tactile bimodal stimulation, suggesting a faster and more effective processing and detection of stimuli if auditory stimulation is included. Overall, these findings provide insights into the relationship between multisensory integration and human behavior and cognition.Matteo MarucciGianluca Di FlumeriGianluca BorghiniNicolina SciaraffaMichele ScandolaEnea Francesco PavoneFabio BabiloniViviana BettiPietro AricòNature PortfolioarticleMedicineRScienceQENScientific Reports, Vol 11, Iss 1, Pp 1-15 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	Medicine R Science Q
spellingShingle	Medicine R Science Q Matteo Marucci Gianluca Di Flumeri Gianluca Borghini Nicolina Sciaraffa Michele Scandola Enea Francesco Pavone Fabio Babiloni Viviana Betti Pietro Aricò The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence
description	Abstract Real-world experience is typically multimodal. Evidence indicates that the facilitation in the detection of multisensory stimuli is modulated by the perceptual load, the amount of information involved in the processing of the stimuli. Here, we used a realistic virtual reality environment while concomitantly acquiring Electroencephalography (EEG) and Galvanic Skin Response (GSR) to investigate how multisensory signals impact target detection in two conditions, high and low perceptual load. Different multimodal stimuli (auditory and vibrotactile) were presented, alone or in combination with the visual target. Results showed that only in the high load condition, multisensory stimuli significantly improve performance, compared to visual stimulation alone. Multisensory stimulation also decreases the EEG-based workload. Instead, the perceived workload, according to the “NASA Task Load Index” questionnaire, was reduced only by the trimodal condition (i.e., visual, auditory, tactile). This trimodal stimulation was more effective in enhancing the sense of presence, that is the feeling of being in the virtual environment, compared to the bimodal or unimodal stimulation. Also, we show that in the high load task, the GSR components are higher compared to the low load condition. Finally, the multimodal stimulation (Visual-Audio-Tactile—VAT and Visual-Audio—VA) induced a significant decrease in latency, and a significant increase in the amplitude of the P300 potentials with respect to the unimodal (visual) and visual and tactile bimodal stimulation, suggesting a faster and more effective processing and detection of stimuli if auditory stimulation is included. Overall, these findings provide insights into the relationship between multisensory integration and human behavior and cognition.
format	article
author	Matteo Marucci Gianluca Di Flumeri Gianluca Borghini Nicolina Sciaraffa Michele Scandola Enea Francesco Pavone Fabio Babiloni Viviana Betti Pietro Aricò
author_facet	Matteo Marucci Gianluca Di Flumeri Gianluca Borghini Nicolina Sciaraffa Michele Scandola Enea Francesco Pavone Fabio Babiloni Viviana Betti Pietro Aricò
author_sort	Matteo Marucci
title	The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence
title_short	The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence
title_full	The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence
title_fullStr	The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence
title_full_unstemmed	The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence
title_sort	impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence
publisher	Nature Portfolio
publishDate	2021
url	https://doaj.org/article/2e1e1e55954b425c8ef20a09f25dfcab
work_keys_str_mv	AT matteomarucci theimpactofmultisensoryintegrationandperceptualloadinvirtualrealitysettingsonperformanceworkloadandpresence AT gianlucadiflumeri theimpactofmultisensoryintegrationandperceptualloadinvirtualrealitysettingsonperformanceworkloadandpresence AT gianlucaborghini theimpactofmultisensoryintegrationandperceptualloadinvirtualrealitysettingsonperformanceworkloadandpresence AT nicolinasciaraffa theimpactofmultisensoryintegrationandperceptualloadinvirtualrealitysettingsonperformanceworkloadandpresence AT michelescandola theimpactofmultisensoryintegrationandperceptualloadinvirtualrealitysettingsonperformanceworkloadandpresence AT eneafrancescopavone theimpactofmultisensoryintegrationandperceptualloadinvirtualrealitysettingsonperformanceworkloadandpresence AT fabiobabiloni theimpactofmultisensoryintegrationandperceptualloadinvirtualrealitysettingsonperformanceworkloadandpresence AT vivianabetti theimpactofmultisensoryintegrationandperceptualloadinvirtualrealitysettingsonperformanceworkloadandpresence AT pietroarico theimpactofmultisensoryintegrationandperceptualloadinvirtualrealitysettingsonperformanceworkloadandpresence AT matteomarucci impactofmultisensoryintegrationandperceptualloadinvirtualrealitysettingsonperformanceworkloadandpresence AT gianlucadiflumeri impactofmultisensoryintegrationandperceptualloadinvirtualrealitysettingsonperformanceworkloadandpresence AT gianlucaborghini impactofmultisensoryintegrationandperceptualloadinvirtualrealitysettingsonperformanceworkloadandpresence AT nicolinasciaraffa impactofmultisensoryintegrationandperceptualloadinvirtualrealitysettingsonperformanceworkloadandpresence AT michelescandola impactofmultisensoryintegrationandperceptualloadinvirtualrealitysettingsonperformanceworkloadandpresence AT eneafrancescopavone impactofmultisensoryintegrationandperceptualloadinvirtualrealitysettingsonperformanceworkloadandpresence AT fabiobabiloni impactofmultisensoryintegrationandperceptualloadinvirtualrealitysettingsonperformanceworkloadandpresence AT vivianabetti impactofmultisensoryintegrationandperceptualloadinvirtualrealitysettingsonperformanceworkloadandpresence AT pietroarico impactofmultisensoryintegrationandperceptualloadinvirtualrealitysettingsonperformanceworkloadandpresence
_version_	1718385439615746048

The impact of multisensory integration and perceptual load in virtual reality settings on performance, workload and presence

Ejemplares similares