|Title:||Cross-Lingual Transfer Learning of Non-Native Acoustic Modeling for Pronunciation Error Detection and Diagnosis|
Kawahara, Tatsuya https://orcid.org/0000-0002-2686-2296 (unconfirmed)
Nanjo, Hiroaki https://orcid.org/0000-0002-6523-6482 (unconfirmed)
|Author's alias:||河原, 達也|
|Keywords:||Speech and Hearing|
Linguistics and Language
Acoustics and Ultrasonics
Electrical and Electronic Engineering
|Publisher:||Institute of Electrical and Electronics Engineers (IEEE)|
|Journal title:||IEEE/ACM Transactions on Audio, Speech, and Language Processing|
|Abstract:||In computer-assisted pronunciation training (CAPT), the scarcity of large-scale non-native corpora and human expert annotations are two fundamental challenges to non-native acoustic modeling. Most existing approaches of acoustic modeling in CAPT are based on non-native corpora while there are so many living languages in the world. It is impractical to collect and annotate every non-native speech corpus considering different language pairs. In this work, we address non-native acoustic modeling (both on phonetic and articulatory level) based on transfer learning. In order to effectively train acoustic models of non-native speech without using such data, we propose to exploit two large native speech corpora of learner's native language (L1) and target language (L2) to model cross-lingual phenomena. This kind of transfer learning can provide a better feature representation of non-native speech. Experimental evaluations are carried out for Japanese speakers learning English. We first demonstrate the proposed acoustic-phone model achieves a lower word error rate in non-native speech recognition. It also improves the pronunciation error detection based on goodness of pronunciation (GOP) score. For diagnosis of pronunciation errors, the proposed acoustic-articulatory modeling method is effective for providing detailed feedback at the articulation level.|
|Rights:||© 2019 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.|
This is not the published version. Please cite only the published version.
|Appears in Collections:||Journal Articles|
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.