Constructing & Evaluating Context-Aware Recommender System in a case study with webshop carts and AB-testing

Abstract

A potential customer spends around five minutes on a webshop and visits roughly ten pages. To maximise sales it is crucial that the most relevant products is presented within those limits. The solution is to use a recommender system that predicts and recommends items on a personal level to the customer using collaborative filtering. Therefore, the project aim was to construct a working prototype for 3Bits and Lindex with an AB-testing phase to validate performance. Such evaluation of the collaborative filtering paradigm is not very common and as it only focus on on-going carts with binary implicit feedback, it does not take other information sources into account. To thoroughly evaluate the performance in this particular configuration five testphases was used: experimental prototype cross-validation tests, prototype testing using historical data sets, prototype testing using contextual pre-filtering, prototype crossvalidation of synthetic generated data and finally AB-testing. A lot of literature and data was studied in order to be able to construct the evaluation tests. Furthermore, to enable AB-testing 3Bits had to integrate the prototype into Lindex web page in a way that maintained the professional level. The evaluation shows a high accuracy compared to recommendations based on the most frequent occurring items. Furthermore, results from AB-tests of the projectalgorithm against the old recommendation services at Lindex showed contradicting results. Additionally, the evaluation also showed that incorporating contextual pre-filtering to the prototype did not increase performance.