A Study of Articulatory Features Based Detection of Pronunciation Erroneous Tendency

doi:10.13209/j.0479-8023.2017.029

Abstract

Abstract:

This paper proposed to apply senone log-likelihood ratio based articulatory features (AFs) to improve pronunciation erroneous tendency (PET) detection performance. The feedback information of articulation-placement and articulation-manner could be derived from the definition of PET. The framework of the method involved two main steps. 1) A bank of attribute extractors based on neural networks were trained to estimate the log-likelihood ratio (LLR) for each senone at a frame level. 2) AFs composed of those LLRs outputted from each attribute extractor were used for detecting PETs. Results demonstrated that the proposed system had better performance than the baseline system using MFCC. Moreover, substantial improvements were obtained by combining AFs with MFCC, achieving a lower false rejection rate of 5.0%, a lower false acceptance rate of 30.8% and a higher diagnostic accuracy of 89.8%.

Key words: articulatory features (AFs), pronunciation erroneous tendency (PET), computer assisted pronuncia-tion training (CAPT), senone log-likelihood ratios

摘要：

为了提升计算机辅助发音训练(CAPT)系统中发音偏误趋势(PET)的检测效果, 确保反馈信息的准确性与有效性, 提出一种基于对数似然比的发音特征方法。该方法将多个基于深度神经网络的发音特征提取器用于生成帧级别的对数似然比, 然后将对数似然比组成的发音特征用于PET的检测, 为学习者提供发音位置和发音方法的正音信息。实验结果表明, 发音特征对PET的检测效果优于常用声学特征(MFCC, PLP和fBank), 当发音特征与MFCC特征相结合时, 可以进一步提升性能, 达到错误接受率为5.0%, 错误拒绝率为30.8%, 诊断正确率为89.8%的检测效果。

关键词: 发音特征, 发音偏误趋势, 计算机辅助发音训练, 对数似然比

CLC Number:

TP391
H193

Leyuan QU, Yanlu XIE, Jinsong ZHANG. A Study of Articulatory Features Based Detection of Pronunciation Erroneous Tendency[J]. Acta Scientiarum Naturalium Universitatis Pekinensis, 2017, 53(2): 239-246.

屈乐园, 解焱陆, 张劲松. 基于发音特征的发音偏误趋势检测研究[J]. 北京大学学报自然科学版, 2017, 53(2): 239-246.

Figures/Tables 10

References 26

[1]	Eskenazi M.An overview of spoken language tech-nology for education. Speech Communication, 2009, 51(10): 832-844
[2]	Dlaska A, Krekeler C.Self-assessment of pronun-ciation. System, 2008, 36(4): 506-516
[3]	Neri A, Cucchiarini C, Strik H, et al.The pedagogy-technology interface in computer assisted pronuncia-tion training. Computer Assisted Language learning, 2002, 15(5): 441-467
[4]	Franco H, Neumeyer L, Ramos M, et al.Automatic detection of phone-level mispronunciation for language learning // EUROSPEECH. Budapest, 1999: 851-854
[5]	Witt S M, Young S J.Phone-level pronunciation scoring and assessment for interactive language learning. Speech Communication, 2000, 30(2): 95-108
[6]	Zhang F, Huang C, Soong F K, et al.Automatic mispronunciation detection for Mandarin // 2008 IEEE International Conference on Acoustics, Speech and Signal Processing. Caesars Palace, 2008: 5077-5080
[7]	Wang Y B, Lee L S.Improved approaches of modeling and detecting error patterns with empirical analysis for computer-aided pronunciation training // 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Kyoto, 2012: 5049-5052
[8]	Hu W, Qian Y, Soong F K, et al. Improved mispronunciation detection with deep neural network trained acoustic models and transfer learning based logistic regression classifiers. Speech Communi-cation, 2015, 67: 154-166
[9]	Witt S M.Automatic error detection in pronunciation training: where we are and where we need to go // Proc ISADEPT. Stockholm, 2012: 12-19
[10]	王岚, 李崇国, 蒙美玲, 等. 音素级错误发音自动检测. 先进技术研究通报, 2009, 3(2): 6-10
[11]	Luo D, Yang X, Wang L.Improvement of segmental mispronunciation detection with prior knowledge extracted from large L2 speech corpus // INTER-SPEECH. Florence, 2011: 1593-1596
[12]	Yoon S Y, Hasegawa-Johnson M, Sproat R.Landmark- based automated pronunciation error detection // INTERSPEECH. Makuhari, 2010: 614-617
[13]	Cao W, Wang D, Zhang J, et al.Developing a Chinese L2 speech database of Japanese learners with narrow-phonetic labels for computer assisted pronunciation training // Eleventh Annual Conference of the Inter-national Speech Communication Association. Chiba, 2010: 1922-1925
[14]	Duan R, Zhang J, Cao W, et al.A preliminary study on ASR-based detection of Chinese mispronunciation by Japanese learners // INTERSPEECH. Singapore, 2014: 1478-1481
[15]	Gao Y, Xie Y, Cao W, et al.A study on robust detection of pronunciation erroneous tendency based on deep neural network // 16th Annual Conference of the International Speech Communication Association (INTERSPEECH). Dresden, 2015: 693-696
[16]	Schroeter J, Sondhi M M.Techniques for estimating vocal-tract shapes from the speech signal. IEEE Transactions on Speech and Audio Processing, 1994, 2(1): 133-150
[17]	Yu D, Siniscalchi S M, Deng L, et al.Boosting attribute and phone estimation accuracies with deep neural networks for detection-based speech reco-gnition // 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Kyoto, 2012: 4169-4172
[18]	Kirchhoff K.Combining articulatory and acoustic information for speech recognition in noisy and reverberant environments // ICSLP. Sydney, 1998: 1-4
[19]	Kirchho K.Robust speech recognition using artic-ulatory information [D]. Bielefeld: University of Bielefeld, 1999
[20]	张晴晴, 潘接林, 颜永红. 基于发音特征的汉语普通话语音声学建模. 声学学报, 2010, 35(2): 254-260
[21]	Metze F. Articulatory features for conversational speech recognition [D]. Karlsruhe: Karlsruhe Univer-sity, 2005
[22]	Livescu K, Cetin O, Hasegawa-Johnson M, et al. Articulatory feature-based methods for acoustic and audio-visual speech recognition: Summary from the 2006 JHU summer workshop // 2007 IEEE Inter-national Conference on Acoustics, Speech and Signal Processing. Pacific Grove, CA, 2007: IV-621-IV-624
[23]	Cetin O, Kantor A, King S, et al. An articulatory feature-based tandem approach and factored observa-tion modeling//2007 IEEE International Conference on Acoustics, Speech and Signal Processing. Pacific Grove, CA, 2007: IV-645-IV-648
[24]	Çetin O, Magimai-Doss M, Livescu K, et al.Monolingual and crosslingual comparison of tandem features derived from articulatory and phone MLPs // IEEE Workshop on Automatic Speech Recognition & Understanding. Kyoto, 2007: 36-41
[25]	黄伯荣, 廖序东. 现代汉语. 北京:高等教育出版社, 2007
[26]	Gao S, Xu B, Zhang H, et al.Update progress of Sinohear: advanced Mandarin LVCSR system at NLPR // INTERSPEECH. Beijing, 2000: 798-801

音段	类别	发音特征
声母	发音方法	塞音、塞擦音、擦音、鼻音、边音、送气、清音、浊音
声母	发音位置	双唇音、唇齿音、舌尖前、舌尖中、舌尖后、舌面前、舌面后
韵母	按口型分	撮口呼、齐齿呼、合口呼、开口呼
韵母	按结构分	单元音、双元音、三元音、前鼻音韵母、后鼻音韵母

音段	类别	发音特征
声母	发音方法	塞音、塞擦音、擦音、鼻音、边音、送气、清音、浊音
声母	发音位置	双唇音、唇齿音、舌尖前、舌尖中、舌尖后、舌面前、舌面后
韵母	按口型分	撮口呼、齐齿呼、合口呼、开口呼
韵母	按结构分	单元音、双元音、三元音、前鼻音韵母、后鼻音韵母

类型	标注符号	偏误举例	说明
舌页化	sh	sh{sh}	sh被发成舌页音
圆唇化	o	e{o}	展唇音被发成圆唇音
后化	-	n{-}	前鼻音近似成后鼻音
短化	;	p{;}	p送气时长较短

类型	标注符号	偏误举例	说明
舌页化	sh	sh{sh}	sh被发成舌页音
圆唇化	o	e{o}	展唇音被发成圆唇音
后化	-	n{-}	前鼻音近似成后鼻音
短化	;	p{;}	p送气时长较短

语料内容	描述
文本	301句日常用语
发音人数	7个日本女生
句子总数	1899
音素总数	26431
每句话平均音素数	14
标注者人数	6
每句话标注者人数	2