A genetic programming approach to finding discrepancies in log files

Abstract

An evolutionary algorithm is used to find discrepancies in log files. From a set of error-free reference logs, a set of regular expression patterns describing the logs’ general structure is generated using genetic programming. The patterns can then be checked against logs containing errors, with the goal being that added, removed and reordered lines are detected. Using a regex-oriented approach allows for grouping lines together even though the contents are not exactly the same in every instance. The approach works well so long as the log files provided do not contain too much noise.