DUET: Dually guided knowledge distillation from explicit feedback

Abstract

Various knowledge distillation (KD) methods for recommender systems have been recently introduced to achieve two goals: (i) obtaining an inference time shorter than the cumbersome model (i.e., teacher) and (ii) providing accuracy higher than the compact model (i.e., student). Despite their success, they solely focus on developing KD methods with implicit feedback. We argue that handling CF with explicit feedback is also crucial, representing the different degrees of user preferences. Towards this goal, we propose a novel KD framework for recommender systems, namely Dually gUided knowlEdge disTillation (DUET). We first observe that explicit feedback is interpreted as two types of user preferences, i.e., pre-use preference and post-use preference. Motivated by such characteristics of explicit feedback, we aim to fuse knowledge from the teacher's pre- and post-use preferences by employing two teachers (i.e., teacher #1 and teacher #2). Teacher #1, trained with pre-use preferences, selects some items among unrated ones. Teacher #2, trained with post-use preferences, determines the soft labels (i.e., predicted post-use preferences) of those items chosen by teacher #1. Finally, the student is trained with both the hard labels (i.e., observed post-use preferences) of rated items and the soft labels (i.e., predicted post-use preferences by teacher #2) of the items selected by teacher #1. Extensive experimental results show that our DUET framework consistently outperforms state-of-the-art KD methods on three benchmark datasets. Notably, it beats RD, CD, DE-RRD, BD, and TD up to 13.6%, 18.6%, 16.8%, 9.6%, and 18.6% in terms of NDCG@10, respectively.