Method for Anomaly Detection using Data Requirements - A practical implementation for field test data

Abstract

Anomaly detection is an important problem across a wide range of industries, where the ability to identify rare and unusual patterns within large volumes of data can have significant operational, financial and security implications. This study presents an understanding of the problem statements and challenges involved in existing data pipelines for managing and analyzing sensor data to identify engine anomalies. To gain a better understanding of these challenges, interviews were conducted within the case company. Industrial engines generate large volumes of field test data through embedded soft ware and engine sensors. Embedded software processes this data efficiently to control devices, monitor performance, and ensure that engines operate reliably under resource constraints. Effective monitoring relies on sensor data, such as temperature, pressure, speed, and fuel consumption, to track engine performance in real time. These data are required to detect faults at an early stage, optimize efficiency, and ensure safe operation. Therefore, an effective domain driven anomaly detection method is developed that relies on data requirement specifications to ensure that the designed model is based on domain knowledge. This thesis focuses on implementing a method for domain driven anomaly detection using data requirements derived from multiple sources, including interviews, technical documentation such as Database CAN files, and field test datasets collected from the case company Volvo Penta. The study aims to bridge the gap between raw sensor data and the definition of meaningful data requirements by distinguishing relevant signal anomalies from data error outliers. Furthermore, a domain driven anomaly detection method is designed as an artifact using the defined data requirements to validate anomaly thresholds and address these challenges. In addition, an analysis of the detected anomalies, while considering both temporal context and multi-signal correlations, demonstrates that the proposed approach can accurately identify and classify outliers as either relevant anomalies or data error outliers. This approach improves the reliability of anomaly detection by reducing false interpretations and ensuring that anomalies are evaluated using well defined validation criteria. Consequently, the derived data requirements support the analysis and validation of the proposed anomaly detection method.