An Alternative Evaluation of the Item-based Collaborative Filtering Using Simulated Online Shopping

Abstract

This paper presents a novel method for evaluating the usefulness of online product recommendation. Previous studies on evaluating recommendation systems have mostly relied on two methods：testing the accuracy of estimating user preferences by recommendation systems, or empirically testing the effectiveness with lab experiments involving human participants. The former does not measure the usefulness directly and hence can be misleading; the latter is expensive in that it requires a working online store system and test participants. In order to address the problems, the proposed approach uses simulation to imitate customer behavior and evaluate the usefulness of recommendation. Models for user behavior and an abstract Internet store are developed for simulation. Actual simulation experiments are performed to illustrate the use of the approach.