Singing Lip Sync Animation System Using Audio Spectrum
- Authors
- Kim N.; Park K.
- Issue Date
- Jan-2020
- Publisher
- Springer
- Keywords
- Audio spectrum; Blendshape activation; Fast Fourier Transform; Singing lip sync animation
- Citation
- Lecture Notes in Electrical Engineering, v.536 LNEE, pp 135 - 140
- Pages
- 6
- Journal Title
- Lecture Notes in Electrical Engineering
- Volume
- 536 LNEE
- Start Page
- 135
- End Page
- 140
- URI
- https://scholarworks.bwise.kr/cau/handle/2019.sw.cau/60414
- DOI
- 10.1007/978-981-13-9341-9_23
- ISSN
- 1876-1100
1876-1119
- Abstract
- When humans sing, the changes in pitch and volume are usually significantly greater than when they speak. The shape and size of the mouth changes dramatically when singing compared with when talking, depending on the strength of the voice. In this study, we propose a new model that effectively generates the change in mouth shape in the vocal environment based on the voice strength estimated by analyzing the audio spectrum. We estimate the voice strength by analyzing the spectrum of the voice using a Fast Fourier Transform-based numerical technique for each frame. We apply the intensity of the estimated voice to the morph targets associated with the mouth shape as the weight of the blendshape. Experimental results show a visually convincing lip-synching animation that changes the mouth shape significantly depending on the pitch and volume of the voice.
- Files in This Item
- There are no files associated with this item.
- Appears in
Collections - College of Software > School of Computer Science and Engineering > 1. Journal Articles
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.