Adaptive Face Blending for Face Replacement System

Abstract

The paper presents a blending method for the face replacement system based on a weight map that is adaptive to pose. First, the geometric landmarks of the face are detected from the image containing the target face in order to proceed with the face replacement. Next, triangulation is performed on the face region according to the distribution of the detected landmark points. If the same process is applied to the reference image, and the affine warping transformation is performed between the corresponding triangular mesh obtained from the reference face and target face, a reference face with the same pose as the target face can be newly generated. Finally, the skin tone of the reference face is matched as the skin tone of the target face, and the face replacement is performed. However, an unnatural boundary is created between the replaced region and the background target face. This boundary can usually be removed with the blending technique. In this paper, we generate a standard weight map from the standard front face model in preprocessing step and generate the pose adaptive weight map corresponding to the pose using the transformation relation between the standard face and the target face. The obtained weight map can adaptively cope with the pose change as compared with the consistently set weight map. A more natural face replacement results, therefore, can be obtained.