Exploring the use of an artificial intelligence chatbot as second language conversation partners

Abstract

This study investigated the appropriateness of using artificially intelligent chatbots as conversation partners for second language (L2) learners. 27 Korean high school and 26 college students had a task-oriented conversation with a text-based chatbot, Mitsuku, for 20 minutes. Chat log data were collected and analyzed quantitatively and qualitatively in terms of the quantity of students’ utterances and their vocabulary levels, along with the degree of conversation task success between the chatbot and its users. Both groups finished their tasks, successfully developing conversations with the chatbot and producing double the expected minimum quantity of utterances, although their performances varied individually. Mitsuku’s vocabulary was deemed appropriate for L2 learners' proficiency. The college students used conversational strategies more appropriately than their high school counterparts. Nevertheless, a sentiment analysis showed that the high school students enjoyed talking with Mitsuku to a greater extent than the college students. These results suggest that the chatbot offers L2 learners substantial opportunities as a conversation partner.