Projects

For a summary of our current research, you can read our Mid-Term Report, written March 2009. The rest of the Projects page will be updated to give more detail about recent results after the spring 2010 reports are written.

S2S's research is based around 11 interrelated projects, which are grouped into 4 themes:
Theme I: Multi-linguistic and comparative research on fine phonetic detail - Projects 1-4
Theme II: Imperfect knowledge/signal - Projects 5-6
Theme III: Beyond short units of speech - Projects 7-8
Theme IV: Exemplars and abstraction - Projects 9-11

Jump to:
Project 1 | Project 2 | Project 3 | Project 4 | Project 5 | Project 6 | Project 7 | Project 8 | Project 9 | Project 10 | Project 11 |



Project 1: Corpora for discovering language-specific and language-general patterns of FPD

BUILDING CORPORA for discovering language-specific and language-general patterns of fine phonetic detail. The S2S partners already use many corpora, ranging from spontaneous conversations between two or more people to tightly-controlled read phrases or isolated words; and from large, multi-speaker collections to just a few tokens of a few structural types from one or two speakers. We will use these corpora when appropriate. However, Project 1 will take S2S further by serving three related functions: infrastructure service provision, corpus-building in a range of languages, and hypothesis-testing.The work will include different languages, different speech styles, and comparisons between languages. The focus can include phonetic reductions and turn taking in conversational speech, morphological and segmental distinctions, and interactions between segments and prosody.

View Project

Working on this project: » Prof John Local » Prof Zdena Palková » Prof Wim van Dommelen » Dr Jan Volín » Dr Sven Mattys » Dr Mirjam Ernestus » Dr Annett Jorschick » Rein Ove Sikveland » Prof Jacques Koreman » Prof Torbjørn Svendsen » Dr Sara Howard » Helena Spilková » Prof Rolf Carlson » Dr Jonas Beskow » Prof David House » Prof Björn Granström » Pavel Vondřička » Dan Brenner » Anton Öttl » Kobie van Krieken » Magdalena Ziková

Project 2: Automatic methods to identify FPD; Corpus-based probabilistic work

Project 2 investigates the distribution of particular types of fine phonetic detail (FPD) in speech. The project will focus on the automatic detection of FPD by means of existing and new statistical and statistical learning techniques. It will support and be supported by Project 1. The main aim is to develop tools for the (relatively) rapid automatic identification of FPD, which can be used by phoneticians and engineers alike.

View Project

Working on this project: » Dr Mirjam Ernestus » Prof Torbjørn Svendsen » Barbara Schuppler » Dr Odette Scharenborg » Dr Jonas Beskow

Project 3: Perceptual salience of FPD 1 - Segmentation of speech into words and phrases

Speech-segmentation research investigates how listeners identify word boundaries in connected speech. Many perceptual/linguistic mechanisms supporting speech segmentation are documented, but evidence for their application to real-life speech is virtually non-existent. Having established the distributional validity and reliability of segmentation cues from spontaneous-speech corpora (Project 1), S2S will fill this gap by testing listeners’ reliance on such cues when hearing spontaneous speech. The aim is to provide the most ecologically valid, empirically supported account of speech segmentation to date, thereby helping to model efficient speech understanding.

View Project

Working on this project: » Dr Sven Mattys » Dr Mirjam Ernestus » Prof Sarah Hawkins » Olesya Rauch » Prof Jeff Bowers » Dr Laurence White

Project 4: Perceptual salience of FPD: Cross-linguistic comparison of contextual sensitivity

Identification of segments and words often requires FPD to be evaluated and potentially re-evaluated in the light of following speech. E.g. bad girl may sound more like bag girl due to regressive place assimilation. The listener must evaluate the /g/ in bag girl relative to the place of articulation of the following consonant. Previous research has shown an exquisite sensitivity to the FPD of assimilation and has highlighted a context-sensitive recognition process for assimilated speech. However, current debate focuses on whether the perceptual system becomes tuned to the assimilation processes specific to a listener’s native language. We will address this question by manipulating both the language of the stimuli and the listener’s native language.

View Project

Working on this project: » Prof Gareth Gaskell » Prof Noël Nguyen » Prof Dr Ulrich Frauenfelder » Prof Dr Oliver Niebuhr » Dr Meghan Clayards » Dr Christine Meunier

Project 5: Perceptual coherence and the role of FPD in noise

This project aims to use computational techniques to build linguistic structures such as prosodic trees on the basis of ‘perceptual coherence’, that is, the grouping of sound components such as harmonics and formants into larger units. The Sheffield project will examine overlap in conversations. The Cluj project will examine other aspects of the relationship between FPD and auditory perceptual coherence, and test the hypothesis that FPD-derived coherence contributes to speech processing in noise.

View Project

Working on this project: » Prof Bill Wells » Dr Guy Brown » Prof Dr Mircea Giurgiu » Jan Gorisch » Ahsanul Kabir » Dr Jonas Beskow » Prof Rolf Carlson » Prof David House » Prof Björn Granström

Project 6: Low-level processing in L2 speech perception

Speech recognition by L2 learners suffers in adverse listening conditions. Several factors are likely to be involved in this performance disparity: (i) interference from the native language; (ii) incomplete acquisition (lack of FPD) of L2 categories; and (iii) presumed universal confusions due to inherent maskability of certain sounds. It is important to tease apart these factors if effective L2 training regimes are to be devised. In addition, knowledge of L2 perception will lead to insights into processes and representations used by native listeners. The purpose of this study is to clarify the role of the above factors via a tightly-linked series of behavioural and computer modelling studies.

View Project

Working on this project: » Dr M. Luisa García Lecumberri » Prof Dr Mircea Giurgiu » Dr Jan Volín » Joanna Sankowska » Prof Martin Cooke » Prof Jasone Cenoz » Dr Radek Skarnitzl » Dr Maria Paola Bissiri

Project 7: Integration of multiple units in computational models

One of the challenges of making use of FPD in ASR is how to incorporate long-term structures which represent speech dynamics and suprasegmental processes into existing recognizers which employ short-term representations.  The frame-based nature of most current work in ASR is at odds with the richer, multiple tree-based representations implied by approaches such as Polysp. Indeed, in ASR, suprasegmental information is typically seen as the source of distracting variation rather than as valuable information. The purpose of this project is to attempt to develop an effective, statistical framework for ASR which is capable of exploiting the information available at multiple time scales. The study has two components. In the first, researchers will build on existing work at Naples and Nijmegen into novel speech feature representations and ASR architectures. The second study is equally adventurous and will examine multimodal FPD.

View Project

Working on this project: » Dr Francesco Cutugno » Dr Jon Barker » Dr Louis ten Bosch » Dr Anna Corazza » Bogdan Ludusan » Dr Gianpaolo Coro » Dr Jonas Beskow » Prof Rolf Carlson » Prof David House » Prof Björn Granström

Project 8: Prosodic structure and FPD: segmental-suprasegmental interaction

The aim of this project is to elucidate how disparate acoustic parameters covary to cause language-specific interpretations of prosodic properties and conversational functions. S2S partners differ widely in their methods. Intonologists use laboratory phonology to produce formal  phonological descriptions of f0. Conversation Analysts study functional roles of correlated phonetic parameters in conversations. The S2S groups have expertise in virtually all paradigms used to explore prosodic structure and its interaction with segments, and will use pairs of languages which are more closely related (Dutch-English, French-Italian) or less closely related (initially Czech, extended if possible to other S2S languages). Intonation systems differ in each of these pairs to a greater or lesser extent. Cross-linguistic comparisons will show to what extent the dependencies are language-specific. 

View Project

Working on this project: » Dr Brechtje Post » Prof MariaPaola D'Imperio » Prof Carlos Gussenhoven » Prof Noël Nguyen » Prof Zdena Palková » Meg Zellers » Francesco Cangemi » Prof David House

Project 9: Hybrid episodic-abstract computational modelling: ASR

Two fundamentals of traditional HMM ASR systems, namely generalization of phoneme-like units and a top-down search procedure, are in conflict with current understanding of HSP, i.e. the preservation of FPD and the importance of bottom-up processing. Properties of recognition of novel words and L2 learning require early abstraction of (sub)-phoneme-like units; while other phenomena hint at detailed segmental matching. Project 9 seeks to further improve its recent combined episodic and abstractionist SR model. The focus for the first half of the project will be on weighting acoustic-phonetic information relative to more abstract knowledge. Later this project may seek to incorporate systematic linguistic knowledge fed to it by Project 10, using the bottom-up information in ways that humans are thought to.

View Project

Working on this project: » Prof Dirk Van Compernolle » Prof Roger Moore » Dr Sébastien Demange » Dr Kris Demuynck » Dr Dino Seppi

Project 10: Hybrid episodic-abstract computational modelling: HSP

Although standard HSP models have had considerable success in simulating some of the basic phenomena of spoken word recognition, any further progress will be critically dependent on constructing models that pay proper attention to the FPD available in the speech signal. This project aims to achieve this by constructing a new computational model that uses FPD to guide the construction of prosodic and grammatical structures that can drive the recognition process. 

View Project

Working on this project: » Dr Dennis Norris » Dr Richard Ogden » Prof Sarah Hawkins » Marco Aldo Piccolino Boniforti

Project 11: Lexical decoding of speech using sub-phonemic features

New: Michele Gubian’s website containing information and tutorials on Functional Data Analysis

This project addresses the lexical coding and decoding of FPD by means of probabilistic subphonemic feature representations that are automatically derived from the speech signal. Subphonemic feature vectors for each 10 millisecond interval in the unfolding speech signal provide an excellent window on the fine phonetic detail across many acoustic and articulatory dimensions. The main aim is to improve computational modelling of HSP by using key techniques from ASR. The starting point will be existing models of ASR and HSP, such as the conventional HMM-based ASR models (Nijmegen), exemplar-based models (Leuven & Sheffield), and the ASR-based model of HSP called SpeM (Nijmegen).

View Project

Working on this project: » Dr Louis ten Bosch » Dr Helmer Strik » Prof Roger Moore » Dr Odette Scharenborg » Dr Michele Gubian

October 2020
S M T W T F S
        1 2 3
4 5 6 7 8 9 10
11 12 13 14 15 16 17
18 19 20 21 22 23 24
25 26 27 28 29 30 31

Marie Curie Logo