One-Shot Speaker Adaptation Based on Initialization by Generative Adversarial Networks for TTS
- Authors
- Lee, Jaeuk; Chang, Joon-Hyuk
- Issue Date
- Sep-2022
- Publisher
- International Speech Communication Association
- Keywords
- multi-speaker; speaker adaptation; voice cloning
- Citation
- Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH, v.2022-September, pp.2978 - 2982
- Indexed
- SCOPUS
- Journal Title
- Proceedings of the Annual Conference of the International Speech Communication Association, INTERSPEECH
- Volume
- 2022-September
- Start Page
- 2978
- End Page
- 2982
- URI
- https://scholarworks.bwise.kr/hanyang/handle/2021.sw.hanyang/173087
- DOI
- 10.21437/Interspeech.2022-934
- ISSN
- 2308-457X
- Abstract
- Speaker adaptation for personalizing text-to-speech (TTS) has become increasingly important. Herein, we propose a novel adaptation using a few seconds of data obtained from an unseen speaker. We first use a speaker embedding lookup table to train a multi-speaker TTS model, wherein each speaker embedding in the lookup table contains information representing a speaker's timbre. We propose an initial embedding predictor that extracts initial embedding suitable for the adaptation of unseen speakers. We use trained speaker embeddings to train the initial embedding predictor. Further, adversarial training is applied to improve the performance. After adversarial training, the initial embedding predictor infers the unseen speaker's initial embedding, and it is fine-tuned. As the initial embedding contains timbre information of the unseen speaker, adaptation is achieved faster and with less data than with conventional methods. We validate the performance with a mean opinion score (MOS) and demonstrate that adaptation is feasible with only 5 s of data.
- Files in This Item
-
Go to Link
- Appears in
Collections - 서울 공과대학 > 서울 융합전자공학부 > 1. Journal Articles
![qrcode](https://api.qrserver.com/v1/create-qr-code/?size=55x55&data=https://scholarworks.bwise.kr/hanyang/handle/2021.sw.hanyang/173087)
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.