Evaluation of Machine Learning Algorithms in Recommender Systems: Candidate Recommender Systems in the Staffing Industry

Abstract

Recommender systems are widely discussed in literature as they provide a solution to problems of information overload in a variety of contexts and application areas. When designing such systems, there are a wide range of options regarding what algorithms, approaches and techniques to use. This study addresses the problem of making key design choices when building candidate recommender systems in the staffing industry. Furthermore, the impact of using a variety of metrics to measure different properties of recommender systems is addressed. The study applies a design research approach, at a company providing an online recruiting platform, in which three different candidate recommender systems are implemented and evaluated. The results show that by varying the design of a candidate recommender system, different properties, such as accuracy, coverage,or diversity of recommendations, can be prioritized. By combining more than one recommender system into a larger system, however, many of the weaknesses of applying any individual approach can be circumvented. Also, broadening the scope of evaluation to include other properties than accuracy increases the ability chose a recommender system that performs in a way that is aligned with the business goals.