Spatio-Temporal Transformer Network for Video Restoration
- Authors
- Kim, Tae Hyun; Sajjadi, Mehdi S. M.; Hirsch, Michael; Schölkopf, Bernhard
- Issue Date
- Sep-2018
- Publisher
- Springer Verlag
- Keywords
- Spatio-temporal flow; Spatio-temporal sampler; Spatio-temporal transformer network; Video deblurring; Video super-resolution
- Citation
- Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), v.11207 LNCS, pp.111 - 127
- Indexed
- SCOPUS
- Journal Title
- Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
- Volume
- 11207 LNCS
- Start Page
- 111
- End Page
- 127
- URI
- https://scholarworks.bwise.kr/hanyang/handle/2021.sw.hanyang/149463
- DOI
- 10.1007/978-3-030-01219-9_7
- ISSN
- 0302-9743
- Abstract
- State-of-the-art video restoration methods integrate optical flow estimation networks to utilize temporal information. However, these networks typically consider only a pair of consecutive frames and hence are not capable of capturing long-range temporal dependencies and fall short of establishing correspondences across several timesteps. To alleviate these problems, we propose a novel Spatio-temporal Transformer Network (STTN) which handles multiple frames at once and thereby manages to mitigate the common nuisance of occlusions in optical flow estimation. Our proposed STTN comprises a module that estimates optical flow in both space and time and a resampling layer that selectively warps target frames using the estimated flow. In our experiments, we demonstrate the efficiency of the proposed network and show state-of-the-art restoration results in video super-resolution and video deblurring.
- Files in This Item
-
Go to Link
- Appears in
Collections - 서울 공과대학 > 서울 컴퓨터소프트웨어학부 > 1. Journal Articles
![qrcode](https://api.qrserver.com/v1/create-qr-code/?size=55x55&data=https://scholarworks.bwise.kr/hanyang/handle/2021.sw.hanyang/149463)
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.