Yichen's Homepage

Hello! I’m currently pursuing my Master’s in Artificial Intelligence Engineering at Carnegie Mellon University, where I’m actively engaged in research at WAVLab under the guidance of Prof. Shinji Watanabe. My primary research areas include Multimodal Language Models, Multimodal Fusion, Speech Recognition, AVSR, and Speech Translation.

I hold a Bachelor’s degree in Computer Science from the University of Illinois at Urbana-Champaign. During my undergraduate studies, I had the opportunity to work as a research assistant in the CyPhy Group, supervised by Prof. Tarek Abdelazher. Additionally, I gained valuable experience collaborating with Prof. Kris Hauser and Prof. Yuxiong Wang in the Intelligent Motion Lab.

My research interests are centered around Multimodal Large Language Models(MLLM) and Multimodal Fusion, with a particular focus on integrating speech and visual modalities.

Google Scholar  Github  Email  CV  LinkedIn  Ins 

News

• [12/2024] Graduated from CMU!
• [12/2024] One paper accepted by AAAI 2025
• [09/2024] FastAdaSP has been accepted by EMNLP2024!
• [07/2024] My first first-author paper has been accepted by Interspeech 2024 Syndata4genai Workshop!
• [01/2024] Released my first open source project: ViDove V0.1.0! See the official intro here
• [08/2023] Joined WAVLab as a research assistant
• [08/2023] Start my journey at CMU!
• [05/2023] Start my internship at Trova.AI, supervised by Sanjay Patel
• [05/2023] Graduated with the highest distinction from UIUC!
• [06/2022] Joined CyPhy Group as an undergraduate research assistant working on robust graph learning on dynamic link prediction.

Publications

• FastAdaSP: Multitask-Adapted Efficient Inference for Large Speech Language Model
  Yichen Lu*, Jiaqi Song*, Chao-Han Huck Yang, Shinji Watanabe
  EMNLP 2024 Industry Track (Oral)
  Paper
  Blog
• Enhancing Audiovisual Speech Recognition through Bifocal Preference Optimization Yihan Wu, Yichen Lu, Yifan Peng, Xihua Wang, Ruihua Song, Shinji Watanabe AAAI 2025
• SynesLM: A Unified Approach for Audio-visual Speech Recognition and Translation via Language Model and Synthetic Data
  Yichen Lu*, Jiaqi Song*, Xuankai Chang, Hengwei Bian, Soumi Maiti, Shinji Watanabe
  Interspeech 2024 Syndata4genai Workshop
  Paper
• Robust Audiovisual Speech Recognition Models with Mixture-of-Experts
  Yihan Wu, Yifan Peng, Yichen Lu, Xuankai Chang, Ruihua Song, Shinji Watanabe
  SLT 2024
  Paper
• Exploring Speech Recognition, Translation, and Understanding with Discrete Speech Units: A Comparative Study
  Xuankai Chang, Brian Yan, Kwanghee Choi, Jeeweon Jung, Yichen Lu, Soumi Maiti, Roshan Sharma, Jiatong Shi, Jinchuan Tian, Shinji Watanabe, Yuya Fujita, Takashi Maekaku, Pengcheng Guo, Yao-Fei Cheng, Pavel Denisov, Kohei Saijo, Hsiu-Hsuan Wang
  ICASSP 2024
  Paper
• Robust Reasoning Over Noisy Knowledge Graphs Via Structure Learning.
  Ruijie Wang, Baoyu Li, Yichen Lu, Tarek Abdelzaher.
  ACL Findings 2023
  Paper

Research Experiences

• [08/2023 - Present] WAVLab, supervised by Prof. Shinji Watanabe
• [06/2022 - 05/2023] CyPhy Group, supervised by Prof. Tarek Abdelazher
• [02/2022 - 08/2022] Intelligent Motion Lab, supervised by Prof. Kris Hauser and Prof. Yuxiong Wang

Industry Experiences

• [05/2023 - 08/2023] Machine Learning Engineer Intern at Trova.AI
• [07/2021 - 01/2022] Software Engineer Intern at VMware, Inc.
• [10/2020 - 02/2021] Software Engineer Intern at NetEase, Inc.