Investigating Semantic Integration of Gestures with Nouns in Neurodiverse Individuals via Event Related Potentials

This publication is included in the Autistic Autism Scholarship Project. one author of this publication is identified as autistic in the project.

About the autistic author marker

Metadata sources used for this reference

OpenAlex Crossref
Learn more about the sources used

Zuniga, T. A., Hughes, M., Poplin, M., McCrory, B., Wu, Y. C., & Modyanova, N. (2026). Investigating Semantic Integration of Gestures with Nouns in Neurodiverse Individuals via Event Related Potentials. Proceedings of the International Symposium on Human Factors and Ergonomics in Health Care, 15(1), 151-155. https://doi.org/10.1177/2327857926151162

Publication date: 19 Aug 2026 Added to AutiHub: 23 Aug 2026 Type: Other Article language: English
Publisher source (DOI) Full text OpenAlex source

This publication is integrated into AutiHub through:

Authors

Publication authors
6
Publication authors identified as autistic
1 / 6 (16.7%)
  1. Tadeo Aviles Zuniga
  2. Mackenzie Hughes
  3. Michael Poplin
  1. Bernadette McCrory
  2. Ying Choon Wu
  3. Nadezhda (Nadya) Modyanova *

Abstract

Iconic hand gestures paired with speech can enhance word comprehension and are candidates for caregiver delivered interventions in rural settings where access to language services is limited. Whether children’s brains process gestures as communicative semantic content, and whether this differs between typically developing and neurodivergent children, is not well characterized. We recorded 20 channel EEG at 500 Hz from 13 participants across four groups (typically developing four year olds, typically developing older children, older neurodivergent children with ASD, ADHD, or both, and adults) during a passive viewing trifecta paradigm in which each spoken noun appeared with a matching or mismatching animation (Time 0) or iconic gesture (Time 1). After FASTER style preprocessing and epoching time-locked to the final target stimulus of each trial, we examined the N400 response over an eight channel centro-parietal and temporal region of interest. In Gesture word last trials, match and mismatch waveforms diverged approximately 300 ms after speech onset with the expected direction in typically developing older children (d = 0.34), neurodivergent older children (d = 0.78), and typical four year olds (d = 0.36), suggesting that gesture and speech pairings yielded the most consistent semantic differentiation across groups, evidence for semantic processing and integration; the largest single effect was observed for Animation in four year olds (d = 1.80). Animation last and gesture last trials did not show the same canonical pattern. Per word amplitudes in the N400 time window identified Goat, Umbrella, and Emoji as more stable items and Lock, Flag, and Rectangle as more variable. These preliminary findings from eight participants and 251 retained epochs support the trifecta paradigm for studying gesture and speech integration across development and motivate larger studies with normed stimuli.

Bibliography cited by this reference

Cited references are imported from external metadata sources when they are available. The list may be partial.

Cited bibliography overview

These indicators describe the bibliography cited by this publication. An author name is counted each time it appears in one cited reference, so the same person can be counted more than once. Names not yet linked to an author already present in AutiHub are treated as unknown, not as non-autistic. Last computed: 12 Sep 2026 01:04.

Cited references
25
With a DOI
23
Without a DOI, from raw bibliography text
2
3 / 25 (12.0%) cited references include at least one author identified as autistic.
References with data to complete
0 / 25 (0.0%)
References with detected author names
25 / 25 (100.0%)
Without detected author names
0
Without a structured title
0
Without a stable identifier
2
References with raw author names still to review
0
References with external metadata lookup issues
0
These indicators apply to cited references displayed on this page, after technical duplicates have been merged. A reference without a DOI can still support author statistics when a title and author names are available.
Author names detected in the cited bibliography
107
From DOI or external metadata
107
From validated raw bibliography text
0
Raw names already validated
0
Raw names still to review
0
16 / 107 (15.0%) names are linked to an author already present in AutiHub. 91 / 107 (85.0%) names are not yet linked.
Names linked to a person identified as autistic
3 / 107 (2.8%)
Calculated across all author names detected in the cited bibliography. Among names linked to an author already present in AutiHub: 3 / 16 (18.8%). Distinct people identified as autistic: 2 / 103 (1.9%).
  1. Elizabeth M. Fitzpatrick , Jonelle Thibert , Viviane Grandpierre , J. Cyne Johnston (2014). How HANDy are baby signs? A systematic review of the impact of gestural communication on typically developing, hearing infants under the age of 36 months . First Language, 34(6), 486-509. SAGE Publications.
    Type: Article DOI: 10.1177/0142723714562864 OpenAlex: https://openalex.org/W1976172905
    Crossref OpenAlex
  2. Chiara Cantiani , Naseem A. Choudhury , Yan H. Yu , Valerie L. Shafer , Richard G. Schwartz , April A. Benasich (2016). From Sensory Perception to Lexical-Semantic Processing: An ERP Study in Non-Verbal Children with Autism . PLOS ONE, 11(8), e0161637. Public Library of Science (PLoS).
    Type: Article DOI: 10.1371/journal.pone.0161637 OpenAlex: https://openalex.org/W2510804014
    Crossref OpenAlex
  3. Elizabeth A. Sheehan , Laura L. Namy , Debra L. Mills (2007). Developmental changes in neural activity to familiar words and gestures . Brain and Language, 101(3), 246-259. Elsevier BV.
    Type: Article DOI: 10.1016/j.bandl.2006.11.008 OpenAlex: https://openalex.org/W1997547202
    Crossref OpenAlex
  4. Naomi K. Caselli , Zed Sevcikova Sehyr , Ariel M. Cohen-Goldberg , Karen Emmorey (2017). ASL-LEX: A lexical database of American Sign Language . Behavior Research Methods, 49(2), 784-801. Springer Science and Business Media LLC.
    Type: Article DOI: 10.3758/s13428-016-0742-0 OpenAlex: https://openalex.org/W2401084598
    Crossref OpenAlex
  5. H. Nolan , R. Whelan , R.B. Reilly (2010). FASTER: Fully Automated Statistical Thresholding for EEG artifact Rejection . Journal of Neuroscience Methods, 192(1), 152-162. Elsevier BV.
    Type: Article DOI: 10.1016/j.jneumeth.2010.07.015 OpenAlex: https://openalex.org/W2023869733
    Crossref OpenAlex
  6. Dunn L. M. (2007). Peabody Picture Vocabulary Test . Peabody Picture Vocabulary Test.
    Type: Other
    Crossref
  7. Yael Leitner (2014). The Co-Occurrence of Autism and Attention Deficit Hyperactivity Disorder in Children – What Do We Know? Frontiers in Human Neuroscience, 8, 268-268. Frontiers Media SA.
    Type: Article DOI: 10.3389/fnhum.2014.00268 OpenAlex: https://openalex.org/W2025127135
    Crossref OpenAlex
  8. Kara D. Federmeier (2007). Thinking ahead: The role and roots of prediction in language comprehension . Psychophysiology, 44(4), 491-505. Wiley.
    Type: Article DOI: 10.1111/j.1469-8986.2007.00531.x OpenAlex: https://openalex.org/W2096575147
    Crossref OpenAlex
  9. Aslı Özyürek (2014). Hearing and seeing meaning in speech and gesture: insights from brain and behaviour . Philosophical Transactions of the Royal Society B: Biological Sciences, 369(1651), 20130296. The Royal Society.
    Type: Article DOI: 10.1098/rstb.2013.0296 OpenAlex: https://openalex.org/W2076807085
    Crossref OpenAlex
  10. Wolfgang Skrandies (1990). Global field power and topographic similarity . Brain Topography, 3(1), 137-141. Springer Science and Business Media LLC.
    Type: Article DOI: 10.1007/bf01128870 OpenAlex: https://openalex.org/W2087849126
    Crossref OpenAlex
  11. Susan Goldin-Meadow , Martha Wagner Alibali (2013). Gesture's Role in Speaking, Learning, and Creating Language . Annual Review of Psychology, 64(1), 257-283. Annual Reviews.
    Type: Article DOI: 10.1146/annurev-psych-113011-143802 OpenAlex: https://openalex.org/W2103763726
    Crossref OpenAlex
  12. Henning Holle , Thomas C. Gunter (2007). The Role of Iconic Gestures in Speech Disambiguation: ERP Evidence . Journal of Cognitive Neuroscience, 19(7), 1175-1192. MIT Press - Journals.
    Type: Article DOI: 10.1162/jocn.2007.19.7.1175 OpenAlex: https://openalex.org/W2108086496
    Crossref OpenAlex
  13. A. Hyvarinen (1999). Fast and robust fixed-point algorithms for independent component analysis . IEEE Transactions on Neural Networks, 10(3), 626-634. Institute of Electrical and Electronics Engineers (IEEE).
    Type: Article DOI: 10.1109/72.761722 OpenAlex: https://openalex.org/W2141224535
    Crossref OpenAlex
  14. Olessia Jouravlev , David Zheng , Zuzanna Balewski , Alvince Le Arnz Pongos , Zena Levan , Susan Goldin-Meadow et al. (2019). Speech-accompanying gestures are not processed by the language-processing mechanisms . Neuropsychologia, 132, 107132. Elsevier BV.
    Type: Article DOI: 10.1016/j.neuropsychologia.2019.107132 OpenAlex: https://openalex.org/W2953794415
    Crossref OpenAlex
  15. Roel M. Willems , Peter Hagoort (2007). Neural evidence for the interplay between language, gesture, and action: A review . Brain and Language, 101(3), 278-289. Elsevier BV.
    Type: Article DOI: 10.1016/j.bandl.2007.03.004 OpenAlex: https://openalex.org/W2136260269
    Crossref OpenAlex
  16. Kaufman A. S. (2004). Kaufman Brief Intelligence Test . Kaufman Brief Intelligence Test.
    Type: Other
    Crossref
  17. Marta Kutas , Kara D. Federmeier (2011). Thirty Years and Counting: Finding Meaning in the N400 Component of the Event-Related Brain Potential (ERP) . Annual Review of Psychology, 62(1), 621-647. Annual Reviews.
    Type: Article DOI: 10.1146/annurev.psych.093008.131123 OpenAlex: https://openalex.org/W2141138276
    Crossref OpenAlex
  18. Spencer D. Kelly , Peter Creigh , James Bartolotti (2010). Integrating Speech and Iconic Gestures in a Stroop-like Task: Evidence for Automatic Processing . Journal of Cognitive Neuroscience, 22(4), 683-694. MIT Press.
    Type: Article DOI: 10.1162/jocn.2009.21254 OpenAlex: https://openalex.org/W2161540120
    Crossref OpenAlex
  19. Matthew J. Maenner , Zachary Warren , Ashley Robinson Williams , Esther Amoakohene , Amanda V. Bakian , Deborah A. Bilder et al. (2023). Prevalence and characteristics of autism spectrum disorder among children aged 8 years—Autism and Developmental Disabilities Monitoring Network, 11 sites, United States, 2020. Surveillance Summaries, 72(2), 1-14. Centers for Disease Control MMWR Office.
    Type: Article DOI: 10.15585/mmwr.ss7202a1 OpenAlex: https://openalex.org/W4360600181
    Crossref OpenAlex
  20. Courtenay Frazier Norbury , Debbie Gooch , Charlotte Wray , Gillian Baird , Tony Charman , Emily Simonoff et al. (2016). The impact of nonverbal ability on prevalence and clinical presentation of language disorder: evidence from a population study . Journal of Child Psychology and Psychiatry, 57(11), 1247-1257. Wiley.
    Type: Article DOI: 10.1111/jcpp.12573 OpenAlex: https://openalex.org/W2406021639
    Crossref OpenAlex
  21. David McNeill (2005). Gesture and Thought . University of Chicago Press.
    Type: Book DOI: 10.7208/chicago/9780226514642.001.0001
    Crossref
  23. Kazuki Sekine , Christina Schoechl , Kimberley Mulder , Judith Holler , Spencer Kelly , Reyhan Furman et al. (2020). Evidence for children’s online integration of simultaneous information from speech and iconic gestures: an ERP study . Language, Cognition and Neuroscience, 35(10), 1283-1294. Informa UK Limited.
    Type: Article DOI: 10.1080/23273798.2020.1737719 OpenAlex: https://openalex.org/W3010968392
    Crossref OpenAlex