Speech Technology for Unwritten Languages

Odette Scharenborg; Lucas Ondel; Shruti Palaskar; Philip Arthur; Francesco Ciannella; Mingxing Du; Elin Larsen; Danny Merkx; Rachid Riad; Liming Wang; Emmanuel Dupoux; Laurent Besacier; Alan Black; Mark Hasegawa-Johnson; Florian Metze; Graham Neubig; Sebastian Stüker; Pierre Godard; Markus Müller

doi:10.1109/TASLP.2020.2973896

Speech Technology for Unwritten Languages

Odette Scharenborg, Lucas Ondel, Shruti Palaskar, Philip Arthur, Francesco Ciannella, Mingxing Du, Elin Larsen, Danny Merkx, Rachid Riad, Liming Wang, Emmanuel Dupoux, Laurent Besacier, Alan Black, Mark Hasegawa-Johnson, Florian Metze, Graham Neubig, Sebastian Stüker, Pierre Godard, Markus Müller

Research output: Contribution to journal › Article › peer-review

Abstract

Speech technology plays an important role in our everyday life. Among others, speech is used for human-computer interaction, for instance for information retrieval and on-line shopping. In the case of an unwritten language, however, speech technology is unfortunately difficult to create, because it cannot be created by the standard combination of pre-trained speech-to-text and text-to-speech subsystems. The research presented in this article takes the first steps towards speech technology for unwritten languages. Specifically, the aim of this work was 1) to learn speech-to-meaning representations without using text as an intermediate representation, and 2) to test the sufficiency of the learned representations to regenerate speech or translated text, or to retrieve images that depict the meaning of an utterance in an unwritten language. The results suggest that building systems that go directly from speech-to-meaning and from meaning-to-speech, bypassing the need for text, is possible.

Original language	English (US)
Article number	8998182
Pages (from-to)	964-975
Number of pages	12
Journal	IEEE/ACM Transactions on Audio Speech and Language Processing
Volume	28
DOIs	https://doi.org/10.1109/TASLP.2020.2973896
State	Published - 2020

Keywords

Speech processing
automatic speech recognition
image retrieval
speech synthesis
unsupervised learning

ASJC Scopus subject areas

Computer Science (miscellaneous)
Acoustics and Ultrasonics
Computational Mathematics
Electrical and Electronic Engineering

Online availability

10.1109/TASLP.2020.2973896

Library availability

Discover UIUC Full Text

Cite this

Scharenborg, O., Ondel, L., Palaskar, S., Arthur, P., Ciannella, F., Du, M., Larsen, E., Merkx, D., Riad, R., Wang, L., Dupoux, E., Besacier, L., Black, A., Hasegawa-Johnson, M., Metze, F., Neubig, G., Stüker, S., Godard, P., & Müller, M. (2020). Speech Technology for Unwritten Languages. IEEE/ACM Transactions on Audio Speech and Language Processing, 28, 964-975. Article 8998182. https://doi.org/10.1109/TASLP.2020.2973896

Scharenborg, O, Ondel, L, Palaskar, S, Arthur, P, Ciannella, F, Du, M, Larsen, E, Merkx, D, Riad, R, Wang, L, Dupoux, E, Besacier, L, Black, A, Hasegawa-Johnson, M, Metze, F, Neubig, G, Stüker, S, Godard, P & Müller, M 2020, 'Speech Technology for Unwritten Languages', IEEE/ACM Transactions on Audio Speech and Language Processing, vol. 28, 8998182, pp. 964-975. https://doi.org/10.1109/TASLP.2020.2973896

@article{375b429243db4ae0bcc3b341eb4d35f4,

title = "Speech Technology for Unwritten Languages",

abstract = "Speech technology plays an important role in our everyday life. Among others, speech is used for human-computer interaction, for instance for information retrieval and on-line shopping. In the case of an unwritten language, however, speech technology is unfortunately difficult to create, because it cannot be created by the standard combination of pre-trained speech-to-text and text-to-speech subsystems. The research presented in this article takes the first steps towards speech technology for unwritten languages. Specifically, the aim of this work was 1) to learn speech-to-meaning representations without using text as an intermediate representation, and 2) to test the sufficiency of the learned representations to regenerate speech or translated text, or to retrieve images that depict the meaning of an utterance in an unwritten language. The results suggest that building systems that go directly from speech-to-meaning and from meaning-to-speech, bypassing the need for text, is possible.",

keywords = "Speech processing, automatic speech recognition, image retrieval, speech synthesis, unsupervised learning",

author = "Odette Scharenborg and Lucas Ondel and Shruti Palaskar and Philip Arthur and Francesco Ciannella and Mingxing Du and Elin Larsen and Danny Merkx and Rachid Riad and Liming Wang and Emmanuel Dupoux and Laurent Besacier and Alan Black and Mark Hasegawa-Johnson and Florian Metze and Graham Neubig and Sebastian St{\"u}ker and Pierre Godard and Markus M{\"u}ller",

note = "Publisher Copyright: {\textcopyright} 2014 IEEE.",

year = "2020",

doi = "10.1109/TASLP.2020.2973896",

language = "English (US)",

volume = "28",

pages = "964--975",

journal = "IEEE/ACM Transactions on Audio Speech and Language Processing",

issn = "2329-9290",

publisher = "IEEE Advancing Technology for Humanity",

}

TY - JOUR

T1 - Speech Technology for Unwritten Languages

AU - Scharenborg, Odette

AU - Ondel, Lucas

AU - Palaskar, Shruti

AU - Arthur, Philip

AU - Ciannella, Francesco

AU - Du, Mingxing

AU - Larsen, Elin

AU - Merkx, Danny

AU - Riad, Rachid

AU - Wang, Liming

AU - Dupoux, Emmanuel

AU - Besacier, Laurent

AU - Black, Alan

AU - Hasegawa-Johnson, Mark

AU - Metze, Florian

AU - Neubig, Graham

AU - Stüker, Sebastian

AU - Godard, Pierre

AU - Müller, Markus

PY - 2020

Y1 - 2020

N2 - Speech technology plays an important role in our everyday life. Among others, speech is used for human-computer interaction, for instance for information retrieval and on-line shopping. In the case of an unwritten language, however, speech technology is unfortunately difficult to create, because it cannot be created by the standard combination of pre-trained speech-to-text and text-to-speech subsystems. The research presented in this article takes the first steps towards speech technology for unwritten languages. Specifically, the aim of this work was 1) to learn speech-to-meaning representations without using text as an intermediate representation, and 2) to test the sufficiency of the learned representations to regenerate speech or translated text, or to retrieve images that depict the meaning of an utterance in an unwritten language. The results suggest that building systems that go directly from speech-to-meaning and from meaning-to-speech, bypassing the need for text, is possible.

AB - Speech technology plays an important role in our everyday life. Among others, speech is used for human-computer interaction, for instance for information retrieval and on-line shopping. In the case of an unwritten language, however, speech technology is unfortunately difficult to create, because it cannot be created by the standard combination of pre-trained speech-to-text and text-to-speech subsystems. The research presented in this article takes the first steps towards speech technology for unwritten languages. Specifically, the aim of this work was 1) to learn speech-to-meaning representations without using text as an intermediate representation, and 2) to test the sufficiency of the learned representations to regenerate speech or translated text, or to retrieve images that depict the meaning of an utterance in an unwritten language. The results suggest that building systems that go directly from speech-to-meaning and from meaning-to-speech, bypassing the need for text, is possible.

KW - Speech processing

KW - automatic speech recognition

KW - image retrieval

KW - speech synthesis

KW - unsupervised learning

UR - http://www.scopus.com/inward/record.url?scp=85079642575&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85079642575&partnerID=8YFLogxK

U2 - 10.1109/TASLP.2020.2973896

DO - 10.1109/TASLP.2020.2973896

M3 - Article

AN - SCOPUS:85079642575

SN - 2329-9290

VL - 28

SP - 964

EP - 975

JO - IEEE/ACM Transactions on Audio Speech and Language Processing

JF - IEEE/ACM Transactions on Audio Speech and Language Processing

M1 - 8998182

ER -

Speech Technology for Unwritten Languages

Abstract

Keywords

ASJC Scopus subject areas

Online availability

Library availability

Related links

Fingerprint

Cite this