Towards Fully Mobile 3D Face, Body, and Environment Capture Using Only Head-worn Cameras
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Cha, Young-Woon | - |
dc.contributor.author | Price, True | - |
dc.contributor.author | Wei, Zhen | - |
dc.contributor.author | Lu, Xinran | - |
dc.contributor.author | Rewkowski, Nicholas | - |
dc.contributor.author | Chabra, Rohan | - |
dc.contributor.author | Qin, Zihe | - |
dc.contributor.author | Kim, Hyounghun | - |
dc.contributor.author | Su, Zhaoqi | - |
dc.contributor.author | Liu, Yebin | - |
dc.contributor.author | Ilie, Adrian | - |
dc.contributor.author | State, Andrei | - |
dc.contributor.author | Xu, Zhenlin | - |
dc.contributor.author | Frahm, Jan-Michael | - |
dc.contributor.author | Fuchs, Henry | - |
dc.date.accessioned | 2022-01-11T06:40:04Z | - |
dc.date.available | 2022-01-11T06:40:04Z | - |
dc.date.created | 2022-01-11 | - |
dc.date.issued | 2018-11 | - |
dc.identifier.issn | 1077-2626 | - |
dc.identifier.uri | https://scholarworks.bwise.kr/gachon/handle/2020.sw.gachon/83238 | - |
dc.description.abstract | We propose a new approach for 3D reconstruction of dynamic indoor and outdoor scenes in everyday environments, leveraging only cameras worn by a user. This approach allows 3D reconstruction of experiences at any location and virtual tours from anywhere. The key innovation of the proposed ego-centric reconstruction system is to capture the wearer's body pose and facial expression from near-body views, e.g. cameras on the user's glasses, and to capture the surrounding environment using outward-facing views. The main challenge of the ego-centric reconstruction, however, is the poor coverage of the near-body views - that is, the user's body and face are observed from vantage points that are convenient for wear but inconvenient for capture. To overcome these challenges, we propose a parametric-model-based approach to user motion estimation. This approach utilizes convolutional neural networks (CNNs) for near-view body pose estimation, and we introduce a CNN-based approach for facial expression estimation that combines audio and video. For each time-point during capture, the intermediate model-based reconstructions from these systems are used to re-target a high-fidelity pre-scanned model of the user. We demonstrate that the proposed self-sufficient, head-worn capture system is capable of reconstructing the wearer's movements and their surrounding environment in both indoor and outdoor situations without any additional views. As a proof of concept, we show how the resulting 3D-plus-time reconstruction can be immersively experienced within a virtual reality system (e.g., the HTC Vive). We expect that the size of the proposed egocentric capture-and-reconstruction system will eventually be reduced to fit within future AR glasses, and will be widely useful for immersive 3D telepresence, virtual tours, and general use-anywhere 3D content creation. | - |
dc.language | 영어 | - |
dc.language.iso | en | - |
dc.publisher | IEEE COMPUTER SOC | - |
dc.relation.isPartOf | IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS | - |
dc.title | Towards Fully Mobile 3D Face, Body, and Environment Capture Using Only Head-worn Cameras | - |
dc.type | Article | - |
dc.type.rims | ART | - |
dc.description.journalClass | 1 | - |
dc.identifier.wosid | 000449077900017 | - |
dc.identifier.doi | 10.1109/TVCG.2018.2868527 | - |
dc.identifier.bibliographicCitation | IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, v.24, no.11, pp.2993 - 3004 | - |
dc.description.isOpenAccess | N | - |
dc.identifier.scopusid | 2-s2.0-85053152977 | - |
dc.citation.endPage | 3004 | - |
dc.citation.startPage | 2993 | - |
dc.citation.title | IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS | - |
dc.citation.volume | 24 | - |
dc.citation.number | 11 | - |
dc.contributor.affiliatedAuthor | Cha, Young-Woon | - |
dc.type.docType | Article; Proceedings Paper | - |
dc.subject.keywordAuthor | Terms Telepresence | - |
dc.subject.keywordAuthor | Ego-centric Vision | - |
dc.subject.keywordAuthor | Motion Capture | - |
dc.subject.keywordAuthor | Convolutional Neural Networks | - |
dc.subject.keywordPlus | TRACKING | - |
dc.relation.journalResearchArea | Computer Science | - |
dc.relation.journalWebOfScienceCategory | Computer Science, Software Engineering | - |
dc.description.journalRegisteredClass | scie | - |
dc.description.journalRegisteredClass | scopus | - |
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.
1342, Seongnam-daero, Sujeong-gu, Seongnam-si, Gyeonggi-do, Republic of Korea(13120)031-750-5114
COPYRIGHT 2020 Gachon University All Rights Reserved.
Certain data included herein are derived from the © Web of Science of Clarivate Analytics. All rights reserved.
You may not copy or re-distribute this material in whole or in part without the prior written consent of Clarivate Analytics.