Effects of spatial congruity on audio-visual multimodal integration

W. A. Teder-Sälejärvi, F. Di Russo, J. J. McDonald, Steven A. Hillyard

Research output: Contribution to journalArticle

162 Citations (Scopus)

Abstract

Spatial constraints on multisensory integration of auditory (A) and visual (V) stimuli were investigated in humans using behavioral and electrophysiological measures. The aim was to find out whether cross-modal interactions between A and V stimuli depend on their spatial congruity, as has been found for multisensory neurons in animal studies (Stein & Meredith, 1993). Randomized sequences of unimodal (A or V) and simultaneous bimodal (AV) stimuli were presented to right- or left-field locations while subjects made speeded responses to infrequent targets of greater intensity that occurred in either or both modalities. Behavioral responses to the bimodal stimuli were faster and more accurate than to the unimodal stimuli for both same-location and different-location AV pairings. The neural basis of this cross-modal facilitation was studied by comparing event-related potentials (ERPs) to the bimodal AV stimuli with the summed ERPs to the unimodal A and V stimuli. These comparisons revealed neural interactions localized to the ventral occipito-temporal cortex (at 190 msec) and to the superior temporal cortical areas (at 260 msec) for both same- and different-location AV pairings. In contrast, ERP interactions that differed according to spatial congruity included a phase and amplitude modulation of visual-evoked activity localized to the ventral occipito-temporal cortex at 100-400 msec and an amplitude modulation of activity localized to the superior temporal region at 260-280 msec. These results demonstrate overlapping but distinctive patterns of multisensory integration for spatially congruent and incongruent AV stimuli.

Original languageEnglish
Pages (from-to)1396-1409
Number of pages14
JournalJournal of Cognitive Neuroscience
Volume17
Issue number9
DOIs
Publication statusPublished - Sep 2005

Fingerprint

Temporal Lobe
Evoked Potentials
Neurons

ASJC Scopus subject areas

  • Behavioral Neuroscience
  • Cognitive Neuroscience
  • Experimental and Cognitive Psychology

Cite this

Effects of spatial congruity on audio-visual multimodal integration. / Teder-Sälejärvi, W. A.; Di Russo, F.; McDonald, J. J.; Hillyard, Steven A.

In: Journal of Cognitive Neuroscience, Vol. 17, No. 9, 09.2005, p. 1396-1409.

Research output: Contribution to journalArticle

Teder-Sälejärvi, W. A. ; Di Russo, F. ; McDonald, J. J. ; Hillyard, Steven A. / Effects of spatial congruity on audio-visual multimodal integration. In: Journal of Cognitive Neuroscience. 2005 ; Vol. 17, No. 9. pp. 1396-1409.
@article{b8ceb29a6ff44ee4b11eb05d5ae85f3a,
title = "Effects of spatial congruity on audio-visual multimodal integration",
abstract = "Spatial constraints on multisensory integration of auditory (A) and visual (V) stimuli were investigated in humans using behavioral and electrophysiological measures. The aim was to find out whether cross-modal interactions between A and V stimuli depend on their spatial congruity, as has been found for multisensory neurons in animal studies (Stein & Meredith, 1993). Randomized sequences of unimodal (A or V) and simultaneous bimodal (AV) stimuli were presented to right- or left-field locations while subjects made speeded responses to infrequent targets of greater intensity that occurred in either or both modalities. Behavioral responses to the bimodal stimuli were faster and more accurate than to the unimodal stimuli for both same-location and different-location AV pairings. The neural basis of this cross-modal facilitation was studied by comparing event-related potentials (ERPs) to the bimodal AV stimuli with the summed ERPs to the unimodal A and V stimuli. These comparisons revealed neural interactions localized to the ventral occipito-temporal cortex (at 190 msec) and to the superior temporal cortical areas (at 260 msec) for both same- and different-location AV pairings. In contrast, ERP interactions that differed according to spatial congruity included a phase and amplitude modulation of visual-evoked activity localized to the ventral occipito-temporal cortex at 100-400 msec and an amplitude modulation of activity localized to the superior temporal region at 260-280 msec. These results demonstrate overlapping but distinctive patterns of multisensory integration for spatially congruent and incongruent AV stimuli.",
author = "Teder-S{\"a}lej{\"a}rvi, {W. A.} and {Di Russo}, F. and McDonald, {J. J.} and Hillyard, {Steven A.}",
year = "2005",
month = "9",
doi = "10.1162/0898929054985383",
language = "English",
volume = "17",
pages = "1396--1409",
journal = "Journal of Cognitive Neuroscience",
issn = "0898-929X",
publisher = "MIT Press Journals",
number = "9",

}

TY - JOUR

T1 - Effects of spatial congruity on audio-visual multimodal integration

AU - Teder-Sälejärvi, W. A.

AU - Di Russo, F.

AU - McDonald, J. J.

AU - Hillyard, Steven A.

PY - 2005/9

Y1 - 2005/9

N2 - Spatial constraints on multisensory integration of auditory (A) and visual (V) stimuli were investigated in humans using behavioral and electrophysiological measures. The aim was to find out whether cross-modal interactions between A and V stimuli depend on their spatial congruity, as has been found for multisensory neurons in animal studies (Stein & Meredith, 1993). Randomized sequences of unimodal (A or V) and simultaneous bimodal (AV) stimuli were presented to right- or left-field locations while subjects made speeded responses to infrequent targets of greater intensity that occurred in either or both modalities. Behavioral responses to the bimodal stimuli were faster and more accurate than to the unimodal stimuli for both same-location and different-location AV pairings. The neural basis of this cross-modal facilitation was studied by comparing event-related potentials (ERPs) to the bimodal AV stimuli with the summed ERPs to the unimodal A and V stimuli. These comparisons revealed neural interactions localized to the ventral occipito-temporal cortex (at 190 msec) and to the superior temporal cortical areas (at 260 msec) for both same- and different-location AV pairings. In contrast, ERP interactions that differed according to spatial congruity included a phase and amplitude modulation of visual-evoked activity localized to the ventral occipito-temporal cortex at 100-400 msec and an amplitude modulation of activity localized to the superior temporal region at 260-280 msec. These results demonstrate overlapping but distinctive patterns of multisensory integration for spatially congruent and incongruent AV stimuli.

AB - Spatial constraints on multisensory integration of auditory (A) and visual (V) stimuli were investigated in humans using behavioral and electrophysiological measures. The aim was to find out whether cross-modal interactions between A and V stimuli depend on their spatial congruity, as has been found for multisensory neurons in animal studies (Stein & Meredith, 1993). Randomized sequences of unimodal (A or V) and simultaneous bimodal (AV) stimuli were presented to right- or left-field locations while subjects made speeded responses to infrequent targets of greater intensity that occurred in either or both modalities. Behavioral responses to the bimodal stimuli were faster and more accurate than to the unimodal stimuli for both same-location and different-location AV pairings. The neural basis of this cross-modal facilitation was studied by comparing event-related potentials (ERPs) to the bimodal AV stimuli with the summed ERPs to the unimodal A and V stimuli. These comparisons revealed neural interactions localized to the ventral occipito-temporal cortex (at 190 msec) and to the superior temporal cortical areas (at 260 msec) for both same- and different-location AV pairings. In contrast, ERP interactions that differed according to spatial congruity included a phase and amplitude modulation of visual-evoked activity localized to the ventral occipito-temporal cortex at 100-400 msec and an amplitude modulation of activity localized to the superior temporal region at 260-280 msec. These results demonstrate overlapping but distinctive patterns of multisensory integration for spatially congruent and incongruent AV stimuli.

UR - http://www.scopus.com/inward/record.url?scp=24744434602&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=24744434602&partnerID=8YFLogxK

U2 - 10.1162/0898929054985383

DO - 10.1162/0898929054985383

M3 - Article

C2 - 16197693

AN - SCOPUS:24744434602

VL - 17

SP - 1396

EP - 1409

JO - Journal of Cognitive Neuroscience

JF - Journal of Cognitive Neuroscience

SN - 0898-929X

IS - 9

ER -