Computer-Assisted Pronunciation

Computer-Aided Second Language Learning through Speech-based Human-Computer Interactions

Human-Computer Communications Laboratory

Department of Systems Engineering and Engineering Management

Chinese University of Hong Kong

Introduction Background Demonstration Publications Researchers

Demonstration

Enunciate Demo - An online computer-assisted pronunciation training system (PC version)
Enunciate is a prototype of a computer-assisted pronunciation training (CAPT) application with automatic mispronunciation detection and diagnosis for English pronunciation improvement. The system utilizes automatic speech recognition (ASR) technology and a pronunciation lexicon extended with common mispronunciations of Chinese learners of English. The common mispronunciations are derived from a phonological analysis of Cantonese and English to predict possible phonetic confusions. These confusions are formalized as a set of rules, which can generate common mispronunciations for any set of English words. Using this extended pronunciation lexicon, the ASR engine is tasked with word pronunciation recognition. Enunciate translates the recognition results into comprehensible feedback by highlighting the mispronounced words and providing a phonetic transcription of both the model pronunciation and the learner’s own pronunciation. Video Demo

mEnunciate Demo - An online computer-assisted pronunciation training system (Android)
mEnunciate is a mobile verion of Enunciate. Video Demo

Using Kinect for Lip-tracking in Computer-aided Pronunciation:
The video demonstrates the possibility of using Kinect to perform lip-tracking. The extracted lip contour will compare to the correct one for pronouncing same prompt. The noise is the background noise captured by real Kinect. Video Demo

MusicSpeak
Users can enter an arbitrary English sentence in the text box and then click SUBMIT. The system generates a musical rhythm according to the input text and displays the output on a new tab. The user can click the PLAY button to listen to the generated rhythm, while the corresponding words are highlighted with the beat in a time-synchronous manner. Users can also move the pointer over any word and check its phonetic transcription. Vowels in the syllable with primary stress are highlighted in red. Video Demo

		Enunciate Demo - An online computer-assisted pronunciation training system (PC version) Enunciate is a prototype of a computer-assisted pronunciation training (CAPT) application with automatic mispronunciation detection and diagnosis for English pronunciation improvement. The system utilizes automatic speech recognition (ASR) technology and a pronunciation lexicon extended with common mispronunciations of Chinese learners of English. The common mispronunciations are derived from a phonological analysis of Cantonese and English to predict possible phonetic confusions. These confusions are formalized as a set of rules, which can generate common mispronunciations for any set of English words. Using this extended pronunciation lexicon, the ASR engine is tasked with word pronunciation recognition. Enunciate translates the recognition results into comprehensible feedback by highlighting the mispronounced words and providing a phonetic transcription of both the model pronunciation and the learner’s own pronunciation. Video Demo
		mEnunciate Demo - An online computer-assisted pronunciation training system (Android) mEnunciate is a mobile verion of Enunciate. Video Demo
		Using Kinect for Lip-tracking in Computer-aided Pronunciation: The video demonstrates the possibility of using Kinect to perform lip-tracking. The extracted lip contour will compare to the correct one for pronouncing same prompt. The noise is the background noise captured by real Kinect. Video Demo
		MusicSpeak Users can enter an arbitrary English sentence in the text box and then click SUBMIT. The system generates a musical rhythm according to the input text and displays the output on a new tab. The user can click the PLAY button to listen to the generated rhythm, while the corresponding words are highlighted with the beat in a time-synchronous manner. Users can also move the pointer over any word and check its phonetic transcription. Vowels in the syllable with primary stress are highlighted in red. Video Demo