Analysing a modified ranking algorithm for exploratory search

Abstract

Exploratory Search is a small emerging field within Information Retrieval, studying a type of searching called exploratory searching. This type of search is directed towards learning and investigating, and has recently started to draw attention. However, the field of Exploratory Search struggles with its methodology. A central problem is the difficulty to measure improvements due to that exploratory searching by definition lacks precise goals. New tools and ideas are therefore often evaluated with user studies. By focusing on describing how tools and ideas work, researchers can avoid the difficulty and contribute to the field. Such an indirect approach allows formulating measures that can be applied to ranked lists, which, in turn, allow using simulations with many benefits. This study showcases the approach. The aim is to determine if a ranking algorithm modification influence the formation of groups in lists of ranked articles returned from an academic search engine. The data are generated by simulated searches and a Linear Mixed Model is used for the analysis. The main covariates represent how the ranking of a standard ranking algorithm is weighted together with the ranking according to two new criteria. The response variable consists of scores on how tightly connected the ranked articles are, with the importance of links decreasing with the depth, and comes from the application of a measure developed in the thesis. The main result is that the level of interconnectedness between high ranking articles can be clearly and statistically significantly influenced by the modification, although the influence varies with the randomly generated queries. While more research is needed, this might be useful for controlling the articles interconnectedness when constructing a search engine. On a different level, the thesis shows how the indirect approach can be applied, that it enables using simulations, and it indicates that the approach can produce results interesting for exploratory searching.