Language-General Auditory-Visual Speech Perception: Thai-English and Japanese-English McGurk Effects

Burnham, Denis and Dodd, Barbara (2018) Language-General Auditory-Visual Speech Perception: Thai-English and Japanese-English McGurk Effects. Multisensory Research, 31 1-2: 79-110. doi:10.1163/22134808-00002590

Author Burnham, Denis
Dodd, Barbara
Title Language-General Auditory-Visual Speech Perception: Thai-English and Japanese-English McGurk Effects
Journal name Multisensory Research   Check publisher's open access policy
ISSN 2213-4808
Publication date 2018-01-01
Sub-type Article (original research)
DOI 10.1163/22134808-00002590
Open Access Status DOI
Volume 31
Issue 1-2
Start page 79
End page 110
Total pages 32
Place of publication Leiden, PA Netherlands
Publisher Brill
Language eng
Subject 3205 Experimental and Cognitive Psychology
2731 Ophthalmology
2809 Sensory Systems
1707 Computer Vision and Pattern Recognition
2805 Cognitive Neuroscience
Abstract Cross-language McGurk Effects are used to investigate the locus of auditory-visual speech integration. Experiment 1 uses the fact that [], as in 'sing', is phonotactically legal in word-final position in English and Thai, but in word-initial position only in Thai. English and Thai language participants were tested for 'n' perception from auditory [m]/visual [] (A[m]V[]) in word-initial and -final positions. Despite English speakers' native language bias to label word-initial [] as 'n', the incidence of 'n' percepts to A[m]V[] was equivalent for English and Thai speakers in final and initial positions. Experiment 2 used the facts that (i) as in 'that' is not present in Japanese, and (ii) English speakers respond more often with 'tha' than 'da' to A[ba]V[ga], but more often with 'di' than 'thi' to A[bi]V[gi]. English and three groups of Japanese language participants (Beginner, Intermediate, Advanced English knowledge) were presented with A[ba]V[ga] and A[bi]V[gi] by an English (Experiment 2a) or a Japanese (Experiment 2b) speaker. Despite Japanese participants' native language bias to perceive 'd' more often than 'th', the four groups showed a similar phonetic level effect of [a]/[i] vowel context × 'th' vs. 'd' responses to A[b]V[g] presentations. In Experiment 2b this phonetic level interaction held, but was more one-sided as very few 'th' responses were evident, even in Australian English participants. Results are discussed in terms of a phonetic plus postcategorical model, in which incoming auditory and visual information is integrated at a phonetic level, after which there are post-categorical phonemic influences.
Keyword Auditory-visual integration
cross-language speech perception
phonetic and phonemic speech perception
the McGurk effect
Q-Index Code C1
Q-Index Status Provisional Code
Institutional Status UQ

Document type: Journal Article
Sub-type: Article (original research)
Collections: UQ Centre for Clinical Research Publications
HERDC Pre-Audit
Version Filter Type
Citation counts: TR Web of Science Citation Count  Cited 1 times in Thomson Reuters Web of Science Article | Citations
Scopus Citation Count Cited 1 times in Scopus Article | Citations
Google Scholar Search Google Scholar
Created: Sun, 24 Dec 2017, 10:27:03 EST