Model-based human motion capture from monocular video sequences

Abstract

Generating motion and capturing motion of an articulated body for computer animation is an expensive and time-consuming task. Conventionally, animators manually generate intermediate frames between key frames, but this task is very labor-intensive. This paper presents a model-based singularity-free automatic-initialization approach to capturing human motion from widely-available, static background monocular video sequences. A 3D human body model is built and projected on a 2D projection plane to find the best fit with the foreground image silhouette. We convert the human motion capture problem into two types of parameter optimization problems: static optimization and dynamic optimization. First, we determine each model body configuration using static optimizations for every input image. Then, to obtain better description of motion, the results from all static optimizations are fed into a dynamic optimization process where the entire sequence of motion is considered for the user-specified motion. The user-specified motion is defined by the user and the final form of the motion they want. A cost function for static optimization is used to estimate the degree of overlapping between the foreground input image silhouette and a projected 31) model body silhouette. The overlapping is computed using computational geometry by converting a set of pixels from the image domain to a polygon in the real projection plane domain. A cost function for dynamic optimization is the user-specified motion based on the static optimization results as well as image fitting. Our method is used to capture various human motions: walking, pushing, kicking, and hand-shaking.