Lingwei Meng

About

---

I am a Ph.D. candidate at Human-Computer Communications Laboratory (HCCL), The Chinese University of Hong Kong (CUHK), supervised by Prof. Helen Meng. Previously, I received my M.Phil. degree from the Institute of Automation, Chinese Academy of Sciences (CASIA), where I was supervised by Prof. Jie Tian. And I received my Bachelor's degree from Harbin Institute of Technology (HIT).

I am currently a research intern at Microsoft Research Asia, focusing on language modeling for speech synthesis and the integration of speech with large language models. I am also working on speech processing and recognition.

🟢   I am currently on the job market. Please drop me emails!

News

---

[Jul 2024]  We present [MELLE], a concise, continuous-value token-based language model for TTS, forgoing cumbersome NAR steps.
[Jun 2024]  Three papers has been accepted to ISCA INTERSPEECH 2024, including one first-authored [paper].
[Jan 2024]  Two papers has been accepted to IEEE ICASSP 2024.
[Nov 2023]  Started an internship at Microsoft Research Asia.
[May 2023]  One first-authored [paper] has been accepted to ISCA INTERSPEECH 2023.
[Feb 2023]  One first-authored [paper] has been accepted to IEEE ICASSP 2023 (Oral).
[Jun 2022]  Two papers have been accepted to ISCA INTERSPEECH 2022.
[Mar 2022]  Our [paper] was selected as an ⭐️ ESI Highly Cited Paper by Web of Science.
[Jan 2022]  We won the 2nd place on the diarization track of the ICASSP 2022 - M2Met Chanllenge. This is our [paper].

Education

---

Since 2021, Ph.D. student, The Chinese University of Hong Kong (CUHK)
2018 - 2021, M.Phil., Pattern Recognition and Intelligent Systems, Institute of Automation, Chinese Academy of Sciences (CASIA)
2014 - 2018, B.Eng., Electrical Engineering, Harbin Institute of Technology (HIT)

Experiences

---

• Since Nov. 2023Research Intern, Natural Language Computing Group, Microsoft Research Asia
  Studying language modeling for speech synthesis; integrating speech with large language models
  Working closely with Dr. Long Zhou, Dr. Shujie Liu, and Dr. Furu Wei

Selected Publications

---

[Google Scholar]

- Speech related:

• MELLE: Autoregressive Speech Synthesis without Vector Quantization   |   [demo]
  Lingwei Meng, Long Zhou, Shujie Liu, Sanyuan Chen, Bing Han, Shujie Hu, Yanqing Liu, Jinyu Li, Sheng Zhao, Xixin Wu,
  Helen Meng, Furu Wei
  arXiv, 2024

• Empowering Whisper as a Joint Multi-Talker and Target-Talker Speech Recognition System
  Lingwei Meng, Jiawen Kang, Yuejiao Wang, Zengrui Jin, Xixin Wu, Xunying Liu, Helen Meng
  ISCA INTERSPEECH, Kos Island, Greece, 2024

• A Sidecar Separator Can Convert a Single-Talker Speech Recognition System to a Multi-Talker One   |   [slides] [video]
  Lingwei Meng, Jiawen Kang, Mingyu Cui, Yuejiao Wang, Xixin Wu, Helen Meng
  IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Rhodes Island, Greece, 2023   (Oral)

• Unified Modeling of Multi-Talker Overlapped Speech Recognition and Diarization with a Sidecar Separator
  Lingwei Meng, Jiawen Kang, Mingyu Cui, Haibin Wu, Xixin Wu, Helen Meng
  ISCA INTERSPEECH, Dublin, Ireland, 2023

• VALL-E R: Robust and Efficient Zero-Shot Text-to-Speech Synthesis via Monotonic Alignment   |   [demo]
  Bing Han, Long Zhou, Shujie Liu, Sanyuan Chen, Lingwei Meng, Yanming Qian, Yanqing Liu, Sheng Zhao, Jinyu Li, Furu Wei
  arXiv, 2024

• Large Language Model-based FMRI Encoding of Language Functions for Subjects with Neurocognitive Disorder
  Yuejiao Wang, Xianmin Gong, Lingwei Meng, Xixin Wu, Helen Meng Sheng Zhao, Jinyu Li, Furu Wei
  ISCA INTERSPEECH, Kos Island, Greece, 2024

• LibriheavyMix: A 20,000-Hour Dataset for Single-Channel Reverberant Multi-Talker Speech Separation, ASR and Speaker Diarization
  Zengrui Jin, Yifan Yang, Mohan Shi, Wei Kang, Xiaoyu Yang, Zengwei Yao, Fangjun Kuang, Liyong Guo, Lingwei Meng, Long Lin, Yong Xu, Shi-Xiong Zhang, Daniel Povey
  ISCA INTERSPEECH, Kos Island, Greece, 2024

• Cross-Speaker Encoding Network for Multi-Talker Speech Recognition
  Jiawen Kang, Lingwei Meng, Mingyu Cui, Haohan Guo, Xixin Wu, Xunying Liu, Helen Meng
  IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Seoul, Korea, 2024

• UNIT-DSR: Dysarthric Speech Reconstruction System Using Speech Unit Normalization
  Yuejiao Wang, Xixin Wu, Disong Wang, Lingwei Meng, Helen Meng
  IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Seoul, Korea, 2024

• The defender's perspective on automatic speaker verification: An overview
  Haibin Wu, Jiawen Kang, Lingwei Meng, Helen Meng, Hung-yi Lee
  International Joint Conferences on Artificial Intelligence (IJCAI) Workshop, Macao SAR, China, 2023

• Spoofing-Aware Speaker Verification by Multi-Level Fusion
  Haibin Wu, Lingwei Meng, Jiawen Kang, Jinchao Li, Xu Li, Xixin Wu, Hung-yi Lee, Helen Meng
  ISCA INTERSPEECH, Incheon, Korea, 2022

• Exploring Linguistic Feature and Model Combination for Speech Recognition Based Automatic AD Detection
  Yi Wang, Tianzi Wang, Zi Ye, Lingwei Meng, Shoukang Hu, Xixin Wu, Xunying Liu, Helen Meng
  ISCA INTERSPEECH, Incheon, Korea, 2022

• The CUHK-Tencent Speaker Diarization System for the ICASSP 2022 Multi-Channel Multi-Party Meeting Transcription Challenge
  Naijun Zheng, Na Li, Xixin Wu, Lingwei Meng, Jiawen Kang, Haibin Wu, Chao Weng, Dan Su, Helen Meng
  IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Singapore, 2022

• Tackling Spoofing-Aware Speaker Verification with Multi-Model Fusion
  Haibin Wu, Jiawen Kang, Lingwei Meng, Yang Zhang, Xixin Wu, Zhiyong Wu, Hung-yi Lee, Helen Meng
  ISCA The Speaker and Language Recognition Workshop (Odyssey), Beijing, China, 2022

- Medical Image Analysis related:

• 2D and 3D CT Radiomic Features Performance Comparison in Characterization of Gastric Cancer: a Multi-Center Study
  Lingwei Meng, Di Dong, Xin Chen, Mengjie Fang, Rongpin Wang, Jing Li, Zaiyi Liu, Jie Tian
  IEEE Journal of Biomedical and Health Informatics (IEEE JBHI), 2021
  (ESI Highly Cited Paper)

• A Deep Learning Prognosis Model Help Alert for COVID-19 Patients at High-Risk of Death: a Multi-Center Study
  Lingwei Meng, Di Dong, Liang Li, Meng Niu, Yan Bai, Meiyun Wang, Xiaoming Qiu, Yunfei Zha, Jie Tian
  IEEE Journal of Biomedical and Health Informatics (IEEE JBHI), 2020

• Noninvasive Model for Predicting Future Ischemic Strokes in Patients with Silent Lacunar Infarction Using Radiomics
  Jiehua Su#, Lingwei Meng#, Di Dong#, Wenyan Zhuo, Jianming Wang, Libin Liu, Yi Qin, Ye Tian, Jie Tian, Zhaohui Li
  BMC Medical Imaging, 2020  (# denotes the co-first authorship.)

Challenges

---

• Rank 2/14IEEE ICASSP 2022 Multi-channel Multi-party Meeting Transcription Challenge (M2MeT) - Track 1

Activities

---

• Serving as a reviewer of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)
• Serving as a reviewer of ISCA INTERSPEECH
• Serving as a reviewer of IEEE Journal of Biomedical and Health Informatics

Teaching Assistance

---

SEEM 3440, Operation Research II
AIST 3510 / SEEM 3510, Human-Computer Interaction

Miscellaneous

---

Some awards on control algorithm and circuit design competitions:

• First Prize (main contributor) Aug 2018, the National Marine Vehicle Design Competition, China
• Second Prize (team leader) Jun 2017, ABU Asia-Pacific Robot Contest (RoboCon), China Division
• Second Prize (team leader) Jul 2016, NXP Cup National University Students Intelligent Car Race, China
• Second Prize (team leader) Sep 2015, the National Undergraduate Electronics Design Contest, China