A neurocomputational analysis of the sound-induced flash illusion

Cristiano Cuppini, Elisa Magosso, Nadia Bolognini, Giuseppe Vallar, Mauro Ursino

Research output: Contribution to journalArticle

21 Citations (Scopus)

Abstract

Perception of the external world is based on the integration of inputs from different sensory modalities. Recent experimental findings suggest that this phenomenon is present in lower-level cortical areas at early processing stages. The mechanisms underlying these early processes and the organization of the underlying circuitries are still a matter of debate. Here, we investigate audiovisual interactions by means of a simple neural network consisting of two layers of visual and auditory neurons. We suggest that the spatial and temporal aspects of audio-visual illusions can be explained within this simple framework, based on two main assumptions: auditory and visual neurons communicate via excitatory synapses; and spatio-temporal receptive fields are different in the two modalities, auditory processing exhibiting a higher temporal resolution, while visual processing a higher spatial acuity. With these assumptions, the model is able: i) to simulate the sound-induced flash fission illusion; ii) to reproduce psychometric curves assuming a random variability in some parameters; iii) to account for other audio-visual illusions, such as the sound-induced flash fusion and the ventriloquism illusions; and iv) to predict that visual and auditory stimuli are combined optimally in multisensory integration. In sum, the proposed model provides a unifying summary of spatio-temporal audio-visual interactions, being able to both account for a wide set of empirical findings, and be a framework for future experiments. In perspective, it may be used to understand the neural basis of Bayesian audio-visual inference.

Original languageEnglish
Pages (from-to)248-266
Number of pages19
JournalNeuroImage
Volume92
DOIs
Publication statusPublished - May 15 2014

Fingerprint

Neurons
Psychometrics
Synapses

Keywords

  • Multisensory integration
  • Neural network modeling
  • Sound-induced flash illusion
  • Spatial processing
  • Temporal processing
  • Visual-auditory illusions

ASJC Scopus subject areas

  • Cognitive Neuroscience
  • Neurology
  • Medicine(all)

Cite this

A neurocomputational analysis of the sound-induced flash illusion. / Cuppini, Cristiano; Magosso, Elisa; Bolognini, Nadia; Vallar, Giuseppe; Ursino, Mauro.

In: NeuroImage, Vol. 92, 15.05.2014, p. 248-266.

Research output: Contribution to journalArticle

Cuppini, Cristiano ; Magosso, Elisa ; Bolognini, Nadia ; Vallar, Giuseppe ; Ursino, Mauro. / A neurocomputational analysis of the sound-induced flash illusion. In: NeuroImage. 2014 ; Vol. 92. pp. 248-266.
@article{efb36236691c4b6897d0d4e06775c32a,
title = "A neurocomputational analysis of the sound-induced flash illusion",
abstract = "Perception of the external world is based on the integration of inputs from different sensory modalities. Recent experimental findings suggest that this phenomenon is present in lower-level cortical areas at early processing stages. The mechanisms underlying these early processes and the organization of the underlying circuitries are still a matter of debate. Here, we investigate audiovisual interactions by means of a simple neural network consisting of two layers of visual and auditory neurons. We suggest that the spatial and temporal aspects of audio-visual illusions can be explained within this simple framework, based on two main assumptions: auditory and visual neurons communicate via excitatory synapses; and spatio-temporal receptive fields are different in the two modalities, auditory processing exhibiting a higher temporal resolution, while visual processing a higher spatial acuity. With these assumptions, the model is able: i) to simulate the sound-induced flash fission illusion; ii) to reproduce psychometric curves assuming a random variability in some parameters; iii) to account for other audio-visual illusions, such as the sound-induced flash fusion and the ventriloquism illusions; and iv) to predict that visual and auditory stimuli are combined optimally in multisensory integration. In sum, the proposed model provides a unifying summary of spatio-temporal audio-visual interactions, being able to both account for a wide set of empirical findings, and be a framework for future experiments. In perspective, it may be used to understand the neural basis of Bayesian audio-visual inference.",
keywords = "Multisensory integration, Neural network modeling, Sound-induced flash illusion, Spatial processing, Temporal processing, Visual-auditory illusions",
author = "Cristiano Cuppini and Elisa Magosso and Nadia Bolognini and Giuseppe Vallar and Mauro Ursino",
year = "2014",
month = "5",
day = "15",
doi = "10.1016/j.neuroimage.2014.02.001",
language = "English",
volume = "92",
pages = "248--266",
journal = "NeuroImage",
issn = "1053-8119",
publisher = "Academic Press Inc.",

}

TY - JOUR

T1 - A neurocomputational analysis of the sound-induced flash illusion

AU - Cuppini, Cristiano

AU - Magosso, Elisa

AU - Bolognini, Nadia

AU - Vallar, Giuseppe

AU - Ursino, Mauro

PY - 2014/5/15

Y1 - 2014/5/15

N2 - Perception of the external world is based on the integration of inputs from different sensory modalities. Recent experimental findings suggest that this phenomenon is present in lower-level cortical areas at early processing stages. The mechanisms underlying these early processes and the organization of the underlying circuitries are still a matter of debate. Here, we investigate audiovisual interactions by means of a simple neural network consisting of two layers of visual and auditory neurons. We suggest that the spatial and temporal aspects of audio-visual illusions can be explained within this simple framework, based on two main assumptions: auditory and visual neurons communicate via excitatory synapses; and spatio-temporal receptive fields are different in the two modalities, auditory processing exhibiting a higher temporal resolution, while visual processing a higher spatial acuity. With these assumptions, the model is able: i) to simulate the sound-induced flash fission illusion; ii) to reproduce psychometric curves assuming a random variability in some parameters; iii) to account for other audio-visual illusions, such as the sound-induced flash fusion and the ventriloquism illusions; and iv) to predict that visual and auditory stimuli are combined optimally in multisensory integration. In sum, the proposed model provides a unifying summary of spatio-temporal audio-visual interactions, being able to both account for a wide set of empirical findings, and be a framework for future experiments. In perspective, it may be used to understand the neural basis of Bayesian audio-visual inference.

AB - Perception of the external world is based on the integration of inputs from different sensory modalities. Recent experimental findings suggest that this phenomenon is present in lower-level cortical areas at early processing stages. The mechanisms underlying these early processes and the organization of the underlying circuitries are still a matter of debate. Here, we investigate audiovisual interactions by means of a simple neural network consisting of two layers of visual and auditory neurons. We suggest that the spatial and temporal aspects of audio-visual illusions can be explained within this simple framework, based on two main assumptions: auditory and visual neurons communicate via excitatory synapses; and spatio-temporal receptive fields are different in the two modalities, auditory processing exhibiting a higher temporal resolution, while visual processing a higher spatial acuity. With these assumptions, the model is able: i) to simulate the sound-induced flash fission illusion; ii) to reproduce psychometric curves assuming a random variability in some parameters; iii) to account for other audio-visual illusions, such as the sound-induced flash fusion and the ventriloquism illusions; and iv) to predict that visual and auditory stimuli are combined optimally in multisensory integration. In sum, the proposed model provides a unifying summary of spatio-temporal audio-visual interactions, being able to both account for a wide set of empirical findings, and be a framework for future experiments. In perspective, it may be used to understand the neural basis of Bayesian audio-visual inference.

KW - Multisensory integration

KW - Neural network modeling

KW - Sound-induced flash illusion

KW - Spatial processing

KW - Temporal processing

KW - Visual-auditory illusions

UR - http://www.scopus.com/inward/record.url?scp=84895935730&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84895935730&partnerID=8YFLogxK

U2 - 10.1016/j.neuroimage.2014.02.001

DO - 10.1016/j.neuroimage.2014.02.001

M3 - Article

VL - 92

SP - 248

EP - 266

JO - NeuroImage

JF - NeuroImage

SN - 1053-8119

ER -