Two-stage cross-fusion network for stereo event-based depth estimation
- Authors
- Ghosh, Dipon Kumar; Jung, Yong Ju
- Issue Date
- May-2024
- Publisher
- PERGAMON-ELSEVIER SCIENCE LTD
- Keywords
- Event camera; Stereo matching; Depth estimation; Feature fusion
- Citation
- EXPERT SYSTEMS WITH APPLICATIONS, v.241
- Journal Title
- EXPERT SYSTEMS WITH APPLICATIONS
- Volume
- 241
- URI
- https://scholarworks.bwise.kr/gachon/handle/2020.sw.gachon/89946
- DOI
- 10.1016/j.eswa.2023.122743
- ISSN
- 0957-4174
1873-6793
- Abstract
- Drawing inspiration from biological retina, event camera uses dynamic vision sensor to asynchronously capture pixel intensity changes. Event camera provides a series of advantages, including high dynamic range, high temporal resolution, lack of motion blur, and low power consumption, which make it suitable candidate for depth estimation, specially in challenging scenarios with fast motion and low-lighting conditions. However, due to the sparse nature of event data, it is troublesome to estimate dense disparity map from them. Although it is plausible to estimate disparity map from edges of a structure from where most events are captured, estimating dense disparity is challenging in the areas where events rarely occur. In this study, we develop a stereo matching network for dense disparity estimation, which effectively fuses reconstructed image features with event to estimate disparity. To this end, we propose a two-stage cross-fusion network (TCFNet) to estimate disparity by fusing event and image features with two cascaded fusion mechanisms. First, we propose cross SPADE fusion, based on the spatially-adaptive denormalization mechanism (SPADE), which modulates event features with image features, and image features with event features. In the second stage, the modulated features are further fused using the cross-gating mechanism, which uses the multi-axis gated MLP (gMLP). Experiments on both real and synthetic datasets indicate that the proposed two-stage cross-fusion network outperforms the state-of-the-art event-based stereo matching networks empirically and visually.
- Files in This Item
- There are no files associated with this item.
- Appears in
Collections - ETC > 1. Journal Articles
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.