Event Detection in Smart Meter Data with Complex Event Processing

Abstract

Smart Grids with their accompanying Smart Meters are increasingly taking over from conventional electrical grids and manually inspected electricity meters. This trend gives rise to large amounts of data being collected every day by electricity service providers. Each Smart Meter may emit several readings each hour and can be located at both customers and producers, as well as throughout the infrastructure in locations such as substations. Through analysing this data, a service provider can respond more swiftly to changes in supply and demand, as well as detect anomalies in the grid and at meters. But the large amount of data that is generated quickly exceeds what could be manually inspected and requires the use of techniques made to analyse large quantities of data. One approach is to analyse the data as it arrives in the form of a data stream, without the need to first save it to permanent memory. Two prominent techniques for analysing data streams are those of stream processing and complex event processing. This thesis is conducted in collaboration with Göteborg Energi. It investigates the performance difference of stream processing and complex event processing for pattern detection in Smart Meter data. The findings are further used to guide the implementation of a pattern that combines both techniques and to support the creation of pattern templates. The tests are conducted on three patterns, with a primary focus on comparing the effects on latency and throughput under different levels of source parallelism in the jobs. The results show that while stream processing has a performance advantage over complex event processing on Smart Grid data, combining the two techniques can achieve comparable performance. Using stream processing as an aggregation step before complex event processing can maintain high performance while offering potential reusability and simplifying the creation of future patterns.