A doctoral position will be available at GIPSA-lab in Grenoble from September 2014. The basic question of this PhD is to attempt to understand how audio-visuo-motor speech units are extracted online in the human brain. Candidates should have a master, some knowledge about speech and cognitive psychology, and ability to master tools for conducting behavioral experiments.

 

Context
Speech Unit(e)s: The multisensory-motor unity of speech
Understanding how speech unites the sensory and motor streams, and how speech units emerge from perceptuo-motor interactions
ERC Advanced Grant, Jean-Luc Schwartz, GIPSA-Lab, Grenoble, France

The Speech Unit(e)s project is focused on the speech unification process associating the auditory, visual and motor streams in the human brain, in an interdisciplinary approach combining cognitive psychology, neurosciences, phonetics (both descriptive and developmental) and computational models. The framework is provided by the “Perception-for-Action-Control Theory (PACT)” developed by the PI (Schwartz et al., 2012).
PACT is a perceptuo-motor theory of speech communication, which connects in a principled way perceptual shaping and motor procedural knowledge in speech multisensory processing. The communication unit in PACT is neither a sound nor a gesture but a perceptually shaped gesture, that is a perceptuo-motor unit. It is characterized by both articulatory coherence – provided by its gestural nature – and perceptual value – necessary for being functional. PACT considers two roles for the perceptuo-motor link in speech perception: online unification of the sensory and motor streams through audio-visuo-motor binding, and offline joint emergence of the perceptual and motor repertoires in speech development.

 

Objectives of the PhD position

The basic question of this PhD is to attempt to understand how audio-visuo-motor speech units are extracted online in the human brain. We recently began to study the role of the visual input and of underlying motor processes in the speech segmentation process. We showed how “articulatory chunking” principles related to inter-articulatory synchrony or underlying jaw cycles could provide a “glue” for sticking together pieces of acoustic information. We also showed that visible opening gestures providing onset cues intervene in segmentation and shift the percept towards the compatible sequence.
In this project, we aim to test whether articulatory and visual representations may interfere on the segmentation and lexical access processes, classically confined to acoustic processes. This will involve three major sets of tasks:
(1) Audio-visual and audio-tactile segmentation. We will exploit classical segmentation paradigms, e.g. in French: “prends la fiche” (“take the form”) vs. “prends l’affiche” (“take the poster”). Segmentation has been extensively studied in the audio domain, with proposals about the role of prosody and temporal structure (e.g. Spinelli et al., 2010). However, the visual input is generally ignored for segmentation and lexical access (though see Fort et al., 2010). We will test the role of visual prosody (Dohen et al., 2004) and study whether a visual input coherent with one vs. the other segmentation could drive lexical access towards one or the other interpretation (“fiche”/”form” vs. “affiche”/”poster”). We will also explore the role of a small tactile pulse applied on the jaw in synchrony with one or the other syllable (“fi” for “fiche”, “la” for “l’affiche”), as a first attempt to study the potential role of motor knowledge in the segmentation process.
(2) Dual task paradigms. We assume that the perceptuo-motor link is important for binding and organization of the speech scene before phonetic decoding. We recently proposed and tested double task paradigms involving cyclic jaw or lip gestures, contrasted with manual gestures as control, and showing that orofacial gestures may provide efficient perturbations in these paradigms (Hadian et al., 2012). We will explore various speech comprehension tasks (e.g. understanding speech in noise, comprehension in a foreign language, syllabic monitoring in adverse conditions) with double tasks paradigms, to assess whether a concurrent motor task could perturb comprehension at some level, and particularly the ability to adequately segment the speech material into adequate units.
(3) Neurophysiological mechanisms. A recent set of neuroscience publications converge on a scenario in which the syllable would emerge as a perceptuo-motor construction (Morillon et al., 2010). We will test this assumption by studying whether the representation of the syllabic rhythm in the neuronal code may be enhanced by the coherent visual input, and decreased by a synchronous but incoherent motor activity in a double task paradigm. This would provide clear data in favor of the role of perceptuo-motor links in the online extraction of speech units, which is a long-standing challenge for speech scientists.
The work will be realized within a speech team in GIPSA-Lab Grenoble (partners in the project: Jean-Luc Schwartz, Christophe Savariaux), in close collaboration with a cognitive psychology team in LPNC Grenoble (partners in the project: Sonia Kandel, Elsa Spinelli) and a neuroscience team in the University of Geneva (led by Anne-Lise Giraud).

 

Practical information
The PhD position is open from September 2014, or slightly later if necessary.
Candidates should have a master, some knowledge about speech and cognitive psychology, and ability to master tools for conducting behavioral experiments.
They must send a CV, together with a letter explaining why they are interested in the project. They should also provide two names (with email addresses) for recommendations about their applications.
This should be send before May 30th to Jean-Luc Schwartz (Cette adresse e-mail est protégée contre les robots spammeurs. Vous devez activer le JavaScript pour la visualiser.). Interviews will be done with preselected candidates. Decision will occur in the following weeks.