Snapbot: Enabling Dynamic Human Robot Interactions for Real-Time Computational Photography

Abstract

Photography remains an expert area requiring right focus, exposure, composition, and even post-processing. Yet, robotic automation can enable precise camera manipulation, focus and exposure adjustment, camera composition, and post-processing by leveraging state-of-the-art computational photography. Existing proposals for robotic photography focus on adjusting camera angles for static portraits or developing image evaluation metrics, thus falling short in capturing dynamic human robot interactions. This paper describes the design and implementation of Snapbot, a human robot interaction system designed specifically for computational photography. Snapbot dynamically detects face and pose for exposure and focus and interactively controls robot arm for camera composition to perform image scoring and enhancing. As perception, control, and computational photography form an end-to-end pipeline, Snapbot promises a new future in which image focus, exposure, composition, and generation can be jointly optimized as a unified process. We have implemented and deployed Snapbot on a UR3 demonstrating the mean image quality score is 1.51× compared to aesthetic visual analysis dataset. We also perform ablation study to analyze the impact of each stage of Snapbot both visually and quantitatively. © 2024 Copyright held by the owner/author(s)