Audiovisual integration as conflict resolution: The conflict of the McGurk illusion

Luis Morís Fernández, Emiliano Macaluso, Salvador Soto-Faraco

Research output: Contribution to journalArticlepeer-review

Abstract

There are two main behavioral expressions of multisensory integration (MSI) in speech; the perceptual enhancement produced by the sight of the congruent lip movements of the speaker, and the illusory sound perceived when a speech syllable is dubbed with incongruent lip movements, in the McGurk effect. These two models have been used very often to study MSI. Here, we contend that, unlike congruent audiovisually (AV) speech, the McGurk effect involves brain areas related to conflict detection and resolution. To test this hypothesis, we used fMRI to measure blood oxygen level dependent responses to AV speech syllables. We analyzed brain activity as a function of the nature of the stimuli—McGurk or non-McGurk—and the perceptual outcome regarding MSI—integrated or not integrated response—in a 2 × 2 factorial design. The results showed that, regardless of perceptual outcome, AV mismatch activated general-purpose conflict areas (e.g., anterior cingulate cortex) as well as specific AV speech conflict areas (e.g., inferior frontal gyrus), compared with AV matching stimuli. Moreover, these conflict areas showed stronger activation on trials where the McGurk illusion was perceived compared with non-illusory trials, despite the stimuli where physically identical. We conclude that the AV incongruence in McGurk stimuli triggers the activation of conflict processing areas and that the process of resolving the cross-modal conflict is critical for the McGurk illusion to arise. Hum Brain Mapp 38:5691–5705, 2017.

Original languageEnglish
Pages (from-to)5691-5705
Number of pages15
JournalHuman Brain Mapping
Volume38
Issue number11
DOIs
Publication statusPublished - Nov 1 2017

Keywords

  • audiovisual speech
  • gyrus cinguli
  • illusion
  • magnetic resonance imaging
  • McGurk illusion
  • multisensory integration
  • prefrontal cortex
  • speech perception

ASJC Scopus subject areas

  • Anatomy
  • Radiological and Ultrasound Technology
  • Radiology Nuclear Medicine and imaging
  • Neurology
  • Clinical Neurology

Fingerprint Dive into the research topics of 'Audiovisual integration as conflict resolution: The conflict of the McGurk illusion'. Together they form a unique fingerprint.

Cite this