Interaction Between Words and Symbolic Gestures as Revealed By N400

Maddalena Fabbri-Destro, Pietro Avanzini, Elisa De Stefani, Alessandro Innocenti, Cristina Campi, Maurizio Gentilucci

Research output: Contribution to journalArticle

11 Citations (Scopus)

Abstract

What happens if you see a person pronouncing the word “go” after having gestured “stop”? Differently from iconic gestures, that must necessarily be accompanied by verbal language in order to be unambiguously understood, symbolic gestures are so conventionalized that they can be effortlessly understood in the absence of speech. Previous studies proposed that gesture and speech belong to a unique communication system. From an electrophysiological perspective the N400 modulation was considered the main variable indexing the interplay between two stimuli. However, while many studies tested this effect between iconic gestures and speech, little is known about the capability of an emblem to modulate the neural response to subsequently presented words. Using high-density EEG, the present study aimed at evaluating the presence of an N400 effect and its spatiotemporal dynamics, in terms of cortical activations, when emblems primed the observation of words. Participants were presented with symbolic gestures followed by a semantically congruent or incongruent verb. A N400 modulation was detected, showing larger negativity when gesture and words were incongruent. The source localization during N400 time window evidenced the activation of different portions of temporal cortex according to the gesture and word congruence. Our data provide further evidence of how the observation of an emblem influences verbal language perception, and of how this interplay is mainly instanced by different portions of the temporal cortex.

Original languageEnglish
Pages (from-to)591-605
Number of pages15
JournalBrain Topography
Volume28
Issue number4
DOIs
Publication statusPublished - Aug 15 2014

Fingerprint

Gestures
Emblems and Insignia
Temporal Lobe
Language
Observation
Electroencephalography
Communication

Keywords

  • Anterior temporal pole
  • High density EEG
  • Medial temporal cortex
  • N400 potential
  • Symbolic gestures
  • Words

ASJC Scopus subject areas

  • Clinical Neurology
  • Anatomy
  • Neurology
  • Radiology Nuclear Medicine and imaging
  • Radiological and Ultrasound Technology

Cite this

Fabbri-Destro, M., Avanzini, P., De Stefani, E., Innocenti, A., Campi, C., & Gentilucci, M. (2014). Interaction Between Words and Symbolic Gestures as Revealed By N400. Brain Topography, 28(4), 591-605. https://doi.org/10.1007/s10548-014-0392-4

Interaction Between Words and Symbolic Gestures as Revealed By N400. / Fabbri-Destro, Maddalena; Avanzini, Pietro; De Stefani, Elisa; Innocenti, Alessandro; Campi, Cristina; Gentilucci, Maurizio.

In: Brain Topography, Vol. 28, No. 4, 15.08.2014, p. 591-605.

Research output: Contribution to journalArticle

Fabbri-Destro, M, Avanzini, P, De Stefani, E, Innocenti, A, Campi, C & Gentilucci, M 2014, 'Interaction Between Words and Symbolic Gestures as Revealed By N400', Brain Topography, vol. 28, no. 4, pp. 591-605. https://doi.org/10.1007/s10548-014-0392-4
Fabbri-Destro M, Avanzini P, De Stefani E, Innocenti A, Campi C, Gentilucci M. Interaction Between Words and Symbolic Gestures as Revealed By N400. Brain Topography. 2014 Aug 15;28(4):591-605. https://doi.org/10.1007/s10548-014-0392-4
Fabbri-Destro, Maddalena ; Avanzini, Pietro ; De Stefani, Elisa ; Innocenti, Alessandro ; Campi, Cristina ; Gentilucci, Maurizio. / Interaction Between Words and Symbolic Gestures as Revealed By N400. In: Brain Topography. 2014 ; Vol. 28, No. 4. pp. 591-605.
@article{4928c86d5ec340c39d1e5a75eb4cd1ed,
title = "Interaction Between Words and Symbolic Gestures as Revealed By N400",
abstract = "What happens if you see a person pronouncing the word “go” after having gestured “stop”? Differently from iconic gestures, that must necessarily be accompanied by verbal language in order to be unambiguously understood, symbolic gestures are so conventionalized that they can be effortlessly understood in the absence of speech. Previous studies proposed that gesture and speech belong to a unique communication system. From an electrophysiological perspective the N400 modulation was considered the main variable indexing the interplay between two stimuli. However, while many studies tested this effect between iconic gestures and speech, little is known about the capability of an emblem to modulate the neural response to subsequently presented words. Using high-density EEG, the present study aimed at evaluating the presence of an N400 effect and its spatiotemporal dynamics, in terms of cortical activations, when emblems primed the observation of words. Participants were presented with symbolic gestures followed by a semantically congruent or incongruent verb. A N400 modulation was detected, showing larger negativity when gesture and words were incongruent. The source localization during N400 time window evidenced the activation of different portions of temporal cortex according to the gesture and word congruence. Our data provide further evidence of how the observation of an emblem influences verbal language perception, and of how this interplay is mainly instanced by different portions of the temporal cortex.",
keywords = "Anterior temporal pole, High density EEG, Medial temporal cortex, N400 potential, Symbolic gestures, Words",
author = "Maddalena Fabbri-Destro and Pietro Avanzini and {De Stefani}, Elisa and Alessandro Innocenti and Cristina Campi and Maurizio Gentilucci",
year = "2014",
month = "8",
day = "15",
doi = "10.1007/s10548-014-0392-4",
language = "English",
volume = "28",
pages = "591--605",
journal = "Brain Topography",
issn = "0896-0267",
publisher = "Kluwer Academic/Human Sciences Press Inc.",
number = "4",

}

TY - JOUR

T1 - Interaction Between Words and Symbolic Gestures as Revealed By N400

AU - Fabbri-Destro, Maddalena

AU - Avanzini, Pietro

AU - De Stefani, Elisa

AU - Innocenti, Alessandro

AU - Campi, Cristina

AU - Gentilucci, Maurizio

PY - 2014/8/15

Y1 - 2014/8/15

N2 - What happens if you see a person pronouncing the word “go” after having gestured “stop”? Differently from iconic gestures, that must necessarily be accompanied by verbal language in order to be unambiguously understood, symbolic gestures are so conventionalized that they can be effortlessly understood in the absence of speech. Previous studies proposed that gesture and speech belong to a unique communication system. From an electrophysiological perspective the N400 modulation was considered the main variable indexing the interplay between two stimuli. However, while many studies tested this effect between iconic gestures and speech, little is known about the capability of an emblem to modulate the neural response to subsequently presented words. Using high-density EEG, the present study aimed at evaluating the presence of an N400 effect and its spatiotemporal dynamics, in terms of cortical activations, when emblems primed the observation of words. Participants were presented with symbolic gestures followed by a semantically congruent or incongruent verb. A N400 modulation was detected, showing larger negativity when gesture and words were incongruent. The source localization during N400 time window evidenced the activation of different portions of temporal cortex according to the gesture and word congruence. Our data provide further evidence of how the observation of an emblem influences verbal language perception, and of how this interplay is mainly instanced by different portions of the temporal cortex.

AB - What happens if you see a person pronouncing the word “go” after having gestured “stop”? Differently from iconic gestures, that must necessarily be accompanied by verbal language in order to be unambiguously understood, symbolic gestures are so conventionalized that they can be effortlessly understood in the absence of speech. Previous studies proposed that gesture and speech belong to a unique communication system. From an electrophysiological perspective the N400 modulation was considered the main variable indexing the interplay between two stimuli. However, while many studies tested this effect between iconic gestures and speech, little is known about the capability of an emblem to modulate the neural response to subsequently presented words. Using high-density EEG, the present study aimed at evaluating the presence of an N400 effect and its spatiotemporal dynamics, in terms of cortical activations, when emblems primed the observation of words. Participants were presented with symbolic gestures followed by a semantically congruent or incongruent verb. A N400 modulation was detected, showing larger negativity when gesture and words were incongruent. The source localization during N400 time window evidenced the activation of different portions of temporal cortex according to the gesture and word congruence. Our data provide further evidence of how the observation of an emblem influences verbal language perception, and of how this interplay is mainly instanced by different portions of the temporal cortex.

KW - Anterior temporal pole

KW - High density EEG

KW - Medial temporal cortex

KW - N400 potential

KW - Symbolic gestures

KW - Words

UR - http://www.scopus.com/inward/record.url?scp=84931565352&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84931565352&partnerID=8YFLogxK

U2 - 10.1007/s10548-014-0392-4

DO - 10.1007/s10548-014-0392-4

M3 - Article

VL - 28

SP - 591

EP - 605

JO - Brain Topography

JF - Brain Topography

SN - 0896-0267

IS - 4

ER -