Automatic Chinese Meme Generation using Deep Neural Networks

Abstract

Internet memes have become widely used by people for online communication and interaction, particularly through social media. Interest in meme-generation research has been increasing rapidly. In this study, we address the problem of meme generation as an image captioning task, which uses an encoder–decoder architecture to generate Chinese meme texts that match image content. First, to train the model on the characteristics of Chinese memes, we collected a dataset of 3,000 meme images with 30,000 corresponding humorous Chinese meme texts. Second, we introduced a Chinese meme generation system that can generate humorous and relevant texts from any given image. Our system used a pre-trained ResNet-50 for image feature extraction and a state-of-the-art transformer-based GPT-2 model to generate Chinese meme texts. Finally, we combined the generated text and images to form common image memes. We performed qualitative evaluations of the generated Chinese meme texts through different user studies. The evaluation results revealed that the Chinese memes generated by our model were indistinguishable from real ones. Author