Research Scientist, Spellbrush
Office: Akihabara, Tokyo, Japan | San Francisco, California, US
Remote Working From: Otakaari, Espoo, Finland 👈 Currently Here!
I am a Research Scientist at Spellbrush, a start-up company
passionate about making lifelike AI-powered Anime Waifus and Husbandos for us Otakus.
I work closely with my producer homies Dr. Junan
Zhang and Prof. Lauri
Juvela on research that would help other music producers. I am
also a co-founder of Amphion,
an
open-source toolkit for Audio, Music, and Speech Generation.
My research interests include:
AI Music
Vocaloid
Voiceroid
You will know what I am talking about if you are also an Anime Otaku ;)
Milestones
2026/09
I engaged in the development of YuE2, a frontier-quality
open-source song generation model.
2026/08
Our team launched NORI_OS, a promotional WebARG for our upcoming
AI Anime Waifu Nori, with more than 10k players, over 1M trailer views, and 5k wishlists on our Steam page.
2026/06
I released PupuM2D, the SOTA music SSL model based on my
intuition as a music producer.
2026/05
Our anti-aliasing paper has been accepted by TASLP.
My first large-scale speech dataset, Emilia, got released.
2023/12
My first paper about neural vocoders got
accepted
by ICASSP 2024.
2023/12
My first attempt at developing a
large-scale open-source project, Amphion, succeeded.
2023/10
My first paper about singing voice processing got
accepted
by ML4Audio @ NeurIPS 2023.
2022/09
I entered Professor Zhizheng Wu's lab as a Research Assistant.
Publications
TASLPAliasing-Free Neural Audio Synthesis
Yicheng
Gu, Junan Zhang, Chaoren Wang, Jerry Li, Zhizheng
Wu, Lauri Juvela
IEEE Transactions on Audio, Speech and Language Processing Preprint /
Code
/
Demo /
Hugging Face TL;DR: We propose the first aliasing-free neural vocoder and codec systems in the whole
research area.
submittedFrequency-Aware Self-Supervised Music Representation Learning
Yicheng
Gu, Junan Zhang, Jerry Li, Zhizheng
Wu, Lauri Juvela
Preprint /
Code
/
Demo /
Hugging Face TL;DR: We propose the SOTA music SSL model based on our intuition as music producers.
TASLPAn Investigation of Time-Frequency Representation Discriminators for High-Fidelity Vocoders
Yicheng
Gu, Xueyao Zhang, Liumeng Xue, Haizhou Li, Zhizheng
Wu IEEE/ACM Transactions on Audio, Speech, and Language Processing Preprint /
Code
/
Demo TL;DR: We propose a continuous wavelet transform-based discriminator for GAN-based
neural vocoders.
ICASSP 2024Multi-Scale Sub-Band Constant-Q Transform Discriminator for High-Fidelity Vocoder
Yicheng
Gu, Xueyao Zhang, Liumeng Xue, Zhizheng Wu IEEE International Conference on Acoustics, Speech and Signal Processing 2024 Preprint /
Code
/
Demo /
Hugging Face TL;DR: We propose a constant-Q transform-based discriminator for GAN-based neural
vocoders.
submittedYuE2: Unifying Symbolic and Audio Music Generation at Frontier Quality
Ruibin Yuan, Jiahao Pan, Junyan Jiang, …, Yicheng Gu,
…, Yike Guo
Ruibin Yuan, Jiahao Pan, Junyan Jiang, Zhiyue Wu, Ziya Zhou, Jiankai Sun, Yizhi Li, Ge Zhang, Yicheng Gu, Zeyue Tian, Junyu Dai, Hanfeng Lin, Kai Li, Shangda Wu,
Xuanjie Liu, Jiaming Wang, Zihan Liu, Yue Wang, Yinghao Ma, Hanzhi Yin, Kangrui Chen, Xinyue Zhang,
Ziyang Ma, Mengqi Liao, Hejia Zhao, Guowei Huang, Chao Yan, Lei Ke, Jianwei Yu, Bei Liu, Joe Guo, Liumeng Xue,
Gus Xia, Wei Xue, Yike Guo
[+30 more] Preprint /
Code /
Demo /
Hugging Face TL;DR: We propose a frontier-quality song generator that writes a symbolic score before
rendering
audio.
SLT 2024Emilia: An Extensive, Multilingual, and Diverse Speech Dataset for Large-Scale Speech
Generation
Haorui He*, Zengqiang Shang*, Chaoren Wang*, Xuyuan Li*, Yicheng Gu, …, Zhizheng Wu (*:
Equal Contribution)
Haorui He*, Zengqiang Shang*, Chaoren Wang*, Xuyuan Li*, Yicheng Gu,
Hua Hua, Liwei Liu, Chen Yang, Jiaqi Li, Peiyang Shi, Yuancheng Wang, Kai Chen, Pengyuan Zhang, Zhizheng Wu (*: Equal Contribution)
[+8 more] IEEE Spoken Language Technology Workshop 2024 Preprint /
Code /
Demo /
Hugging Face TL;DR: We propose a large-scale multilingual speech dataset for TTS.
TASLPEmilia: A Large-Scale, Extensive, Multilingual, and Diverse Dataset for Speech Generation
Haorui He*, Zengqiang Shang*, Chaoren Wang*, Xuyuan Li*, Yicheng Gu, …, Zhizheng Wu (*:
Equal Contribution)
Haorui He*, Zengqiang Shang*, Chaoren Wang*, Xuyuan Li*, Yicheng Gu,
Hua Hua, Liwei Liu, Chen Yang, Jiaqi Li, Peiyang Shi, Yuancheng Wang, Kai Chen, Pengyuan Zhang, Zhizheng Wu (*: Equal Contribution)
[+8 more] IEEE Transactions on Audio, Speech and Language Processing Preprint /
Code /
Demo /
Hugging Face TL;DR: We propose version 2.0 of Emilia for TTS.
IJCVFoleyCrafter: Bring Silent Videos to Life with Lifelike and Synchronized Sounds
Yiming Zhang, Yicheng Gu,
Yanhong Zeng, Zhening Xing,
Yuancheng
Wang,
Zhizheng Wu, Bin Liu,
Kai Chen International Journal of Computer Vision Preprint /
Code /
Demo /
Hugging Face TL;DR: We propose a video-to-audio generation pipeline with audio-visual
synchronization and text editability.
Interspeech 2025Neurodyne: Neural Pitch Manipulation with Representation Learning and Cycle-Consistency GAN
Yicheng Gu, Chaoren Wang, Zhizheng Wu, Lauri Juvela
Conference of the International Speech Communication Association 2025 Preprint /
Code /
Demo TL;DR: We propose the SOTA neural autotune model, benchmarked against Melodyne.
DAFx 2025Solid State Bus-Comp: A Large-Scale and Diverse Dataset for Dynamic Range Compressor Virtual
Analog Modeling
Yicheng Gu, Runsong Zhang, Lauri Juvela, Zhizheng Wu International Conference on Digital Audio Effects 2025 Preprint /
Hugging Face /
Demo TL;DR: We propose the first large-scale and diverse dataset for virtual analog
modeling.
submittedMIDIBack: Harmony-Aware Singing Pitch Correction via Joint Vocal-Accompaniment Symbolic
Modeling
Joaquim Cavalcante, Yicheng Gu, Adriel Trajano, Yuri de Malheiros, Thais
Gaudencio
Preprint /
Demo TL;DR: We propose a harmony-aware singing pitch correction model using vocal and
accompaniment
notes.
ICASSP 2026Nord-Parl-TTS: Finnish and Swedish TTS Dataset from Parliament Speech
Zirui Li, Jens Edlund, Yicheng Gu, Nhan Phan, Lauri Juvela, Mikko Kurimo
IEEE International Conference on Acoustics, Speech and Signal Processing 2026 Preprint /
Demo /
Hugging Face TL;DR: We propose an extensive, multilingual, and diverse TTS dataset for Nordic
languages.
submittedHiFi-Glot: High-Fidelity Neural Formant Synthesis with Differentiable Resonant Filters
Yicheng
Gu, Pablo Pérez Zarazaga, Chaoren Wang, Zhizheng
Wu, Zofia Malisz, Gustav Eje Henter, Lauri Juvela
Preprint /
Code
/
Demo TL;DR: We propose a high-fidelity neural formant synthesis system based on
differentiable resonant filters.
ML4Audio @ NeurIPS 2023Leveraging Content-based Features from Multiple Acoustic Models for Singing Voice Conversion
Xueyao Zhang, Yicheng
Gu, Haopeng Chen, Zihao Fang, Lexiao Zou, Liumeng Xue, Zhizheng Wu Machine Learning for Audio Workshop (ML4Audio) at NeurIPS 2023 Paper /
Code /
Demo /
Pretrained Model /
Hugging Face Space /
OpenXLab App TL;DR: We propose to utilize multiple content features for singing voice conversion.
SLT 2024Leveraging Diverse Semantic-based Audio Pretrained Models for Singing Voice Conversion
Xueyao Zhang, Zihao Fang, Yicheng Gu, Haopeng Chen, Lexiao Zou, Junan Zhang, Liumeng Xue, Zhizheng Wu IEEE Spoken Language Technology Workshop 2024 Preprint /
Code /
Demo TL;DR: We investigate the pros and cons of different semantic tokens for singing voice
conversion.
arXivSingNet: Towards a Large-Scale, Diverse, and In-the-Wild Singing Voice Dataset
Yicheng Gu, Chaoren Wang, Junan Zhang, Xueyao Zhang, Zihao Fang, Haorui He,
Zhizheng Wu Preprint /
Demo TL;DR: We propose an extensive, multilingual, and diverse singing voice corpus.
Love Finland
Love Japan
Support Sexual Minorities
,
an
open-source toolkit for Audio, Music, and Speech Generation.