About Sound To Sense

Sound to Sense (S2S): making sense of speech sounds

 

Background

S2S is an interdisciplinary EC-funded Marie Curie Research Training Network (MC-RTN) involving engineers, computer scientists, psychologists, and linguistic phoneticians.

We use a variety of approaches to investigate what types of information are available in the speech signal, and how listeners use that information when they are listening in their native language, or in a foreign language, or in a noisy place like a railway station, when it is hard to hear the speech. These three types of listening situation allow us to see how listeners actively use their knowledge, together with the speech they hear, to understand a message.

Recent research shows that quite fine phonetic detail in the speech signal can carry information crucial to successfully understanding every aspect of a message, from its formal linguistic content, like words and grammar, to the interactional structure which keeps a conversation going. This is not the traditional view, and it challenges most models of speech processing, especially in the central role they give to phonemes and syllables. In contrast, two of S2S’s fundamental principles are that phonetic information is encoded in units of different lengths and degrees of complexity, and that any given sound in the signal fulfils multiple communicative functions simultaneously—its fine detail indicating what those functions are.

Aims

S2S aims to extend and deepen our understanding of the roles phonetic detail plays when people talk with one another. In particular, we will investigate how phonetic detail is used when listeners have:

  • appropriate linguistic-phonetic knowledge (listening to their native language)
  • inappropriate knowledge (listening to a foreign language)
  • inadequate information (listening in noisy environments)

We will do this by modelling, in contexts that reflect both linguistic structure and interactional function:

  • linguistic and statistical properties of language units of different types and durations
  • the interplay between the sensory signal and more abstract knowledge

Cross-disciplinary training is essential to achieve these scientific aims: no single discipline or pair of disciplines currently teaches all the necessary information and skills. So we also aim to:

  • overcome interdisciplinary barriers to progress in speech research, by exchanging knowledge and skills to integrate engineering/computer science, psychology and linguistic phonetics
  • establish a network of scientists who share such skills and will actively collaborate on speech-related research when S2S has finished

The ultimate aim is to develop a model of speech perception that closely reflects the flexibility and robustness of human speech recognition. Such a model would:

  • help pave the way for the next generation of robust automatic speech recognition and text-to-speech machines
  • offer new insights into the diagnosis and treatment of speech disorders
  • offer a new theoretical basis for foreign language teaching

 

Some facts and figures about S2S

  • S2S is a €2.8m Marie Curie Research Training Network funded by the European Community from May 2007 to April 2011.
  • The core of the research is  being done in 14 universities (see Homepage for locations) in 11 countries.
  • A total of 25 training fellowships have been held by young researchers in these universities, and even more senior scientists are involved. Most young researchers have now finished their appointments, and are either writing up PhD theses, or have moved into new jobs.
  • We have agreements with industrial groups who can offer our students internships. More are welcome to join. Write to the Coordinator if you are interested.
  • We hold Workshops every few months. Some of these are open to the public. Anyone with a professional interest in the topics is welcome to register for those that are.
October 2021
S M T W T F S
          1 2
3 4 5 6 7 8 9
10 11 12 13 14 15 16
17 18 19 20 21 22 23
24 25 26 27 28 29 30
31            

Marie Curie Logo