Log-Based Anomaly Detection for System Surveillance

Examensarbete för masterexamen

Please use this identifier to cite or link to this item: https://hdl.handle.net/20.500.12380/219089
Download file(s):
File Description SizeFormat 
219089.pdfFulltext2.84 MBAdobe PDFView/Open
Type: Examensarbete för masterexamen
Master Thesis
Title: Log-Based Anomaly Detection for System Surveillance
Authors: Andreasson, Josefina
Geijer, Cecilia
Abstract: As log files increase in size, it becomes increasingly difficult to manually detect errors within them. There is a need for automated tools for anomaly detection that do not require human assistance. This thesis aims to develop a prototype for such a tool that can be used to monitor the system state based on the produced log files. A specific and a generic approach for analyzing the data is explored to form a foundation for design decisions. Insights from the approaches are then used to build the prototype, which is done in three stages consisting of a basic prototype, extension of the prototype, and evaluation. The prototype is evaluated based on a number of interviews as well as through finding its accuracy and performance. The resulting prototype graphs total lines, words and bigrams per hour. It visualizes the words, bigrams and anomalous messages that occur in each log file. A user specified blacklist highlights undesired words in any file. Anomaly detection is done by comparing historical and current values while taking the overall trends into account. The prototype was found to be useful by two professionals whose work involve log handling, and the interface was thought to be functional. It is able to correctly handle most data but suffers from false alarms, and found 11 out of 14 known errors. A shift in normality is handled well, and the prototype adapts within a week. In conclusion, the developed prototype is usable, mainly for large log files. It requires more accurate anomaly detection, and the interface can be further improved.
Keywords: Data- och informationsvetenskap;Informations- och kommunikationsteknik;Computer and Information Science;Information & Communication Technology
Issue Date: 2015
Publisher: Chalmers tekniska högskola / Institutionen för data- och informationsteknik (Chalmers)
Chalmers University of Technology / Department of Computer Science and Engineering (Chalmers)
URI: https://hdl.handle.net/20.500.12380/219089
Collection:Examensarbeten för masterexamen // Master Theses



Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.