|
Journal Articles
|
TASLP
Aliasing-Free Neural Audio Synthesis
Yicheng
Gu, Junan Zhang, Chaoren Wang, Jerry Li, Zhizheng
Wu, Lauri Juvela
IEEE Transactions on Audio, Speech and Language Processing
Preprint /
Code
/
Demo /
Hugging Face
TL;DR: We propose the first aliasing-free neural vocoder and codec systems in the whole
research area.
|
TASLP
An Investigation of Time-Frequency Representation Discriminators for High-Fidelity Vocoders
Yicheng
Gu, Xueyao Zhang, Liumeng Xue, Haizhou Li, Zhizheng
Wu
IEEE/ACM Transactions on Audio, Speech, and Language Processing
Preprint /
Code
/
Demo
TL;DR: We propose a continuous wavelet transform-based discriminator for GAN-based
neural vocoders.
|
TASLP
Emilia: A Large-Scale, Extensive, Multilingual, and Diverse Dataset for Speech Generation
Haorui He*, Zengqiang Shang*, Chaoren Wang*, Xuyuan Li*, Yicheng Gu, …, Zhizheng Wu (*:
Equal Contribution)
Haorui He*, Zengqiang Shang*, Chaoren Wang*, Xuyuan Li*, Yicheng Gu,
Hua Hua, Liwei Liu, Chen Yang, Jiaqi Li, Peiyang Shi, Yuancheng Wang, Kai Chen, Pengyuan Zhang, Zhizheng Wu (*: Equal Contribution)
[+8 more]
IEEE Transactions on Audio, Speech and Language Processing
Preprint /
Code /
Demo /
Hugging Face
TL;DR: We propose version 2.0 of Emilia for TTS.
|
IJCV
FoleyCrafter: Bring Silent Videos to Life with Lifelike and Synchronized Sounds
Yiming Zhang, Yicheng Gu,
Yanhong Zeng, Zhening Xing,
Yuancheng
Wang,
Zhizheng Wu, Bin Liu,
Kai Chen
International Journal of Computer Vision
Preprint /
Code /
Demo /
Hugging Face
TL;DR: We propose a video-to-audio generation pipeline with audio-visual
synchronization and text editability.
|
|
Conference and Workshop Papers
|
ICASSP 2024
Multi-Scale Sub-Band Constant-Q Transform Discriminator for High-Fidelity Vocoder
Yicheng
Gu, Xueyao Zhang, Liumeng Xue, Zhizheng Wu
IEEE International Conference on Acoustics, Speech and Signal Processing 2024
Preprint /
Code
/
Demo /
Hugging Face
TL;DR: We propose a constant-Q transform-based discriminator for GAN-based neural
vocoders.
|
SLT 2024
Amphion: An Open-Source Audio, Music, and Speech Generation Toolkit
Xueyao Zhang*, Liumeng Xue*,
Yicheng Gu*, Yuancheng Wang*, Jiaqi
Li*, …, Zhizheng Wu (*: Equal Contribution)
Xueyao Zhang*, Liumeng Xue*, Yicheng Gu*, Yuancheng
Wang*, Jiaqi Li*, Haorui He, Chaoren Wang, Songting Liu, Xi Chen, Junan Zhang, Zihao Fang,
Haopeng Chen, Tze Ying Tang, Lexiao Zou, Mingxuan Wang, Jun Han, Kai
Chen, Haizhou Li, Zhizheng
Wu (*: Equal Contribution)
[+13 more]
IEEE Spoken Language Technology Workshop 2024
Preprint /
Code /
Hugging Face
TL;DR: We develop a unified open-source toolkit for audio generation.
|
SLT 2024
Emilia: An Extensive, Multilingual, and Diverse Speech Dataset for Large-Scale Speech
Generation
Haorui He*, Zengqiang Shang*, Chaoren Wang*, Xuyuan Li*, Yicheng Gu, …, Zhizheng Wu (*:
Equal Contribution)
Haorui He*, Zengqiang Shang*, Chaoren Wang*, Xuyuan Li*, Yicheng Gu,
Hua Hua, Liwei Liu, Chen Yang, Jiaqi Li, Peiyang Shi, Yuancheng Wang, Kai Chen, Pengyuan Zhang, Zhizheng Wu (*: Equal Contribution)
[+8 more]
IEEE Spoken Language Technology Workshop 2024
Preprint /
Code /
Demo /
Hugging Face
TL;DR: We propose a large-scale multilingual speech dataset for TTS.
|
Interspeech 2025
Neurodyne: Neural Pitch Manipulation with Representation Learning and Cycle-Consistency GAN
Yicheng Gu, Chaoren Wang, Zhizheng Wu, Lauri Juvela
Conference of the International Speech Communication Association 2025
Preprint /
Demo
TL;DR: We propose the SOTA neural autotune model, benchmarked against Melodyne.
|
DAFx 2025
Solid State Bus-Comp: A Large-Scale and Diverse Dataset for Dynamic Range Compressor Virtual
Analog Modeling
Yicheng Gu*, Runsong Zhang*, Lauri Juvela, Zhizheng Wu (*: Equal Contribution)
International Conference on Digital Audio Effects 2025
Preprint /
Hugging Face /
Demo
TL;DR: We propose the first large-scale and diverse dataset for virtual analog
modeling.
|
ICASSP 2026
Nord-Parl-TTS: Finnish and Swedish TTS Dataset from Parliament Speech
Zirui Li, Jens Edlund, Yicheng Gu, Nhan Phan, Lauri Juvela, Mikko Kurimo
IEEE International Conference on Acoustics, Speech and Signal Processing 2026
Preprint /
Demo /
Hugging Face
TL;DR: We propose an extensive, multilingual, and diverse TTS dataset for Nordic
languages.
|
ML4Audio @ NeurIPS 2023
Leveraging Content-based Features from Multiple Acoustic Models for Singing Voice Conversion
Xueyao Zhang, Yicheng
Gu, Haopeng Chen, Zihao Fang, Lexiao Zou, Liumeng Xue, Zhizheng Wu
Machine Learning for Audio Workshop (ML4Audio) at NeurIPS 2023
Paper /
Code /
Demo /
Pretrained Model
TL;DR: We propose to utilize multiple content features for singing voice conversion.
|
SLT 2024
Leveraging Diverse Semantic-based Audio Pretrained Models for Singing Voice Conversion
Xueyao Zhang, Zihao Fang, Yicheng Gu, Haopeng Chen, Lexiao Zou, Junan Zhang, Liumeng Xue, Zhizheng Wu
IEEE Spoken Language Technology Workshop 2024
Preprint /
Code /
Demo
TL;DR: We investigate the pros and cons of different semantic tokens for singing voice
conversion.
|
|
Preprints
|
Submitted
Frequency-Aware Self-Supervised Music Representation Learning
Yicheng
Gu, Junan Zhang, Jerry Li, Zhizheng
Wu, Lauri Juvela
Preprint /
Code
/
Demo /
Hugging Face
TL;DR: We propose the SOTA music SSL model based on our intuition as music producers.
|
Submitted
YuE2: Unifying Symbolic and Audio Music Generation at Frontier Quality
Ruibin Yuan*, Jiahao Pan*, Junyan Jiang*, …, Yicheng
Gu, …, Yike Guo (*: Equal Contribution)
Ruibin Yuan*, Jiahao Pan*, Junyan Jiang*, Zhiyue Wu*, Ziya Zhou*, Jiankai Sun*, Yizhi Li*, Ge Zhang*,
Yicheng Gu, Zeyue Tian, Junyu Dai, Hanfeng Lin, Kai Li, Shangda Wu,
Xuanjie Liu, Jiaming Wang, Zihan Liu, Yue Wang, Yinghao Ma, Hanzhi Yin, Kangrui Chen, Xinyue Zhang,
Ziyang Ma, Mengqi Liao, Hejia Zhao, Guowei Huang, Chao Yan, Lei Ke, Jianwei Yu, Bei Liu, Joe Guo, Liumeng Xue,
Gus Xia, Wei Xue, Yike Guo (*: Equal Contribution)
[+30 more]
Preprint /
Code /
Demo /
Hugging Face
TL;DR: We propose a frontier-quality song generator that writes a symbolic score before
rendering
audio.
|
Submitted
MIDIBack: Harmony-Aware Singing Pitch Correction via Joint Vocal-Accompaniment Symbolic
Modeling
Joaquim Cavalcante, Yicheng Gu, Adriel Trajano, Yuri de Malheiros, Thais
Gaudencio
Preprint /
Demo
TL;DR: We propose a harmony-aware singing pitch correction model using vocal and
accompaniment
notes.
|
arXiv
HiFi-Glot: High-Fidelity Neural Formant Synthesis with Differentiable Resonant Filters
Yicheng
Gu, Pablo Pérez Zarazaga, Chaoren Wang, Zhizheng
Wu, Zofia Malisz, Gustav Eje Henter, Lauri Juvela
Preprint /
Code
/
Demo
TL;DR: We propose a high-fidelity neural formant synthesis system based on
differentiable resonant filters.
|
arXiv
SingNet: Towards a Large-Scale, Diverse, and In-the-Wild Singing Voice Dataset
Yicheng Gu, Chaoren Wang, Junan Zhang, Xueyao Zhang, Zihao Fang, Haorui He,
Zhizheng Wu
Preprint
TL;DR: We propose an extensive, multilingual, and diverse singing voice corpus.
|
|
Spellbrush
|
Research Scientist & Artist, @San Francisco, USA & Tokyo, Japan (2025/06 ~ now)
#Speech and Singing Voice Synthesis
|
|
The Chinese University of Hong Kong, Shenzhen
|
Research Assistant, @Shenzhen, China (2022/10 ~ 2026/06)
#Speech and Singing Voice Synthesis
|
|
Aalto University
|
Visiting Scholar, @Espoo, Finland (2024/09 ~ 2025/06, 2026/09 ~ now)
#Digital Audio Effects
|
|
Shanghai AI Laboratory
|
Research Assistant, @Shanghai, China (2023/12 ~ 2025/03)
#Video to Audio Generation
|
|
Moises
|
Research Collaborator, @Utah, USA (2025/09 ~ now)
#Neural Autotune
|
|
Reviewer
|
Conferences:
- ICASSP 2025, 2026, 2027
- Interspeech 2026
- ICLR 2025
- IJCNN 2025, 2026, 2027
Journals:
- IEEE Transactions on Audio, Speech and Language Processing (TASLP)
- IEEE Signal Processing Letters (SPL)
|
|
