Surmising synchrony of sound and sight: Factors explaining variance of audiovisual integration in hurdling, tap dancing and drumming.
Auditory and visual percepts are integrated even when they are not perfectly temporally aligned with each other, especially when the visual signal precedes the auditory signal. This window of temporal integration for asynchronous audiovisual stimuli is relatively well examined in the case of speech,...
Guardado en:
| Autores principales: | , , , , , , , , , |
|---|---|
| Formato: | article |
| Lenguaje: | EN |
| Publicado: |
Public Library of Science (PLoS)
2021
|
| Materias: | |
| Acceso en línea: | https://doaj.org/article/354b539a259846809c1e89ee5754451d |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| id |
oai:doaj.org-article:354b539a259846809c1e89ee5754451d |
|---|---|
| record_format |
dspace |
| spelling |
oai:doaj.org-article:354b539a259846809c1e89ee5754451d2021-12-02T20:04:55ZSurmising synchrony of sound and sight: Factors explaining variance of audiovisual integration in hurdling, tap dancing and drumming.1932-620310.1371/journal.pone.0253130https://doaj.org/article/354b539a259846809c1e89ee5754451d2021-01-01T00:00:00Zhttps://doi.org/10.1371/journal.pone.0253130https://doaj.org/toc/1932-6203Auditory and visual percepts are integrated even when they are not perfectly temporally aligned with each other, especially when the visual signal precedes the auditory signal. This window of temporal integration for asynchronous audiovisual stimuli is relatively well examined in the case of speech, while other natural action-induced sounds have been widely neglected. Here, we studied the detection of audiovisual asynchrony in three different whole-body actions with natural action-induced sounds-hurdling, tap dancing and drumming. In Study 1, we examined whether audiovisual asynchrony detection, assessed by a simultaneity judgment task, differs as a function of sound production intentionality. Based on previous findings, we expected that auditory and visual signals should be integrated over a wider temporal window for actions creating sounds intentionally (tap dancing), compared to actions creating sounds incidentally (hurdling). While percentages of perceived synchrony differed in the expected way, we identified two further factors, namely high event density and low rhythmicity, to induce higher synchrony ratings as well. Therefore, we systematically varied event density and rhythmicity in Study 2, this time using drumming stimuli to exert full control over these variables, and the same simultaneity judgment tasks. Results suggest that high event density leads to a bias to integrate rather than segregate auditory and visual signals, even at relatively large asynchronies. Rhythmicity had a similar, albeit weaker effect, when event density was low. Our findings demonstrate that shorter asynchronies and visual-first asynchronies lead to higher synchrony ratings of whole-body action, pointing to clear parallels with audiovisual integration in speech perception. Overconfidence in the naturally expected, that is, synchrony of sound and sight, was stronger for intentional (vs. incidental) sound production and for movements with high (vs. low) rhythmicity, presumably because both encourage predictive processes. In contrast, high event density appears to increase synchronicity judgments simply because it makes the detection of audiovisual asynchrony more difficult. More studies using real-life audiovisual stimuli with varying event densities and rhythmicities are needed to fully uncover the general mechanisms of audiovisual integration.Nina HeinsJennifer PompDaniel S KlugerStefan VinbrüxIma TremplerAxel KohlerKatja KornyshevaKaren ZentgrafMarkus RaabRicarda I SchubotzPublic Library of Science (PLoS)articleMedicineRScienceQENPLoS ONE, Vol 16, Iss 7, p e0253130 (2021) |
| institution |
DOAJ |
| collection |
DOAJ |
| language |
EN |
| topic |
Medicine R Science Q |
| spellingShingle |
Medicine R Science Q Nina Heins Jennifer Pomp Daniel S Kluger Stefan Vinbrüx Ima Trempler Axel Kohler Katja Kornysheva Karen Zentgraf Markus Raab Ricarda I Schubotz Surmising synchrony of sound and sight: Factors explaining variance of audiovisual integration in hurdling, tap dancing and drumming. |
| description |
Auditory and visual percepts are integrated even when they are not perfectly temporally aligned with each other, especially when the visual signal precedes the auditory signal. This window of temporal integration for asynchronous audiovisual stimuli is relatively well examined in the case of speech, while other natural action-induced sounds have been widely neglected. Here, we studied the detection of audiovisual asynchrony in three different whole-body actions with natural action-induced sounds-hurdling, tap dancing and drumming. In Study 1, we examined whether audiovisual asynchrony detection, assessed by a simultaneity judgment task, differs as a function of sound production intentionality. Based on previous findings, we expected that auditory and visual signals should be integrated over a wider temporal window for actions creating sounds intentionally (tap dancing), compared to actions creating sounds incidentally (hurdling). While percentages of perceived synchrony differed in the expected way, we identified two further factors, namely high event density and low rhythmicity, to induce higher synchrony ratings as well. Therefore, we systematically varied event density and rhythmicity in Study 2, this time using drumming stimuli to exert full control over these variables, and the same simultaneity judgment tasks. Results suggest that high event density leads to a bias to integrate rather than segregate auditory and visual signals, even at relatively large asynchronies. Rhythmicity had a similar, albeit weaker effect, when event density was low. Our findings demonstrate that shorter asynchronies and visual-first asynchronies lead to higher synchrony ratings of whole-body action, pointing to clear parallels with audiovisual integration in speech perception. Overconfidence in the naturally expected, that is, synchrony of sound and sight, was stronger for intentional (vs. incidental) sound production and for movements with high (vs. low) rhythmicity, presumably because both encourage predictive processes. In contrast, high event density appears to increase synchronicity judgments simply because it makes the detection of audiovisual asynchrony more difficult. More studies using real-life audiovisual stimuli with varying event densities and rhythmicities are needed to fully uncover the general mechanisms of audiovisual integration. |
| format |
article |
| author |
Nina Heins Jennifer Pomp Daniel S Kluger Stefan Vinbrüx Ima Trempler Axel Kohler Katja Kornysheva Karen Zentgraf Markus Raab Ricarda I Schubotz |
| author_facet |
Nina Heins Jennifer Pomp Daniel S Kluger Stefan Vinbrüx Ima Trempler Axel Kohler Katja Kornysheva Karen Zentgraf Markus Raab Ricarda I Schubotz |
| author_sort |
Nina Heins |
| title |
Surmising synchrony of sound and sight: Factors explaining variance of audiovisual integration in hurdling, tap dancing and drumming. |
| title_short |
Surmising synchrony of sound and sight: Factors explaining variance of audiovisual integration in hurdling, tap dancing and drumming. |
| title_full |
Surmising synchrony of sound and sight: Factors explaining variance of audiovisual integration in hurdling, tap dancing and drumming. |
| title_fullStr |
Surmising synchrony of sound and sight: Factors explaining variance of audiovisual integration in hurdling, tap dancing and drumming. |
| title_full_unstemmed |
Surmising synchrony of sound and sight: Factors explaining variance of audiovisual integration in hurdling, tap dancing and drumming. |
| title_sort |
surmising synchrony of sound and sight: factors explaining variance of audiovisual integration in hurdling, tap dancing and drumming. |
| publisher |
Public Library of Science (PLoS) |
| publishDate |
2021 |
| url |
https://doaj.org/article/354b539a259846809c1e89ee5754451d |
| work_keys_str_mv |
AT ninaheins surmisingsynchronyofsoundandsightfactorsexplainingvarianceofaudiovisualintegrationinhurdlingtapdancinganddrumming AT jenniferpomp surmisingsynchronyofsoundandsightfactorsexplainingvarianceofaudiovisualintegrationinhurdlingtapdancinganddrumming AT danielskluger surmisingsynchronyofsoundandsightfactorsexplainingvarianceofaudiovisualintegrationinhurdlingtapdancinganddrumming AT stefanvinbrux surmisingsynchronyofsoundandsightfactorsexplainingvarianceofaudiovisualintegrationinhurdlingtapdancinganddrumming AT imatrempler surmisingsynchronyofsoundandsightfactorsexplainingvarianceofaudiovisualintegrationinhurdlingtapdancinganddrumming AT axelkohler surmisingsynchronyofsoundandsightfactorsexplainingvarianceofaudiovisualintegrationinhurdlingtapdancinganddrumming AT katjakornysheva surmisingsynchronyofsoundandsightfactorsexplainingvarianceofaudiovisualintegrationinhurdlingtapdancinganddrumming AT karenzentgraf surmisingsynchronyofsoundandsightfactorsexplainingvarianceofaudiovisualintegrationinhurdlingtapdancinganddrumming AT markusraab surmisingsynchronyofsoundandsightfactorsexplainingvarianceofaudiovisualintegrationinhurdlingtapdancinganddrumming AT ricardaischubotz surmisingsynchronyofsoundandsightfactorsexplainingvarianceofaudiovisualintegrationinhurdlingtapdancinganddrumming |
| _version_ |
1718375532706398208 |