Learning to rank, a supervised approach for ranking of documents

Abstract

As available information gets more accessible everywhere and as the rate of new information grows very fast, the systems and models which retrieve this information deserves more attention. The purpose of this thesis is to investigate state-of-the-art machine learning methods for ranking known as learning to rank. The goal is to explore if learning to rank can be used in enterprise search, which means less data and less document features than web based search. Comparisons between several state-of-the-art algorithms from RankLib (Dang, 2011) was carried out on benchmark datasets. Further, Fidelity Loss Ranking (Tsai et al., 2007) was implemented and added to RankLib. The performance of the tests showed that the machine learning algorithms in RankLib had similar performance and that the size of the training sets and the number of features were crucial. Learning to rank is a possible alternative to the standard ranking models in enterprise search only if there are enough features and enough training data. Advise for an implementation of learning to rank in Apache Solr is given, which can be useful for future development. Such an implementation requires a lot of understanding about how the Lucene core works on a low level.