Scheduling algorithms for improved CI performance

Abstract

Continuous Integration (CI) is a software engineering practice aimed at making it easier to integrate code from many developers in a shared code base. An important part of this process is automated building and testing of new code, resulting in various jobs that should be executed on servers controlled through a CI system.

This project evaluates the performance of seven algorithms for job scheduling in a Jenkins based CI environment. Using five separate performance metrics, we look at the trade-offs between different algorithms and discuss their relevance to the experience of using the CI service. Evaluations were mainly carried out in a simulated environment, modeling the Jenkins server, build servers and workloads. The obtained results are then compared to a smaller data set generated on a real Jenkins server, verifying the validity of the simulated results.

The results indicate that the performance of the scheduler can be improved by using algorithms based on known data about the jobs being executed. By selecting a suitable algorithm, it is possible to reduce the cost, as defined by the chosen cost model, compared to the default first-in-first-out strategy used by Jenkins. Comparing the simulated results against data from a real Jenkins server shows that the simulated model is a good representation of the real system, strengthening the validity of the obtained results.