Edge Computing Targeted Data Profiling for Assessing the Quality of Sensor Data

Abstract

This thesis investigated how the data quality of sensor data is determined and if it could be determined efficiently via a framework deployed on an edge computing device. The purpose of this was to enable data quality assessment to be done locally or close to the source of sensor data, i.e., utilize edge computing, in order to overcome the limitations of relying on cloud computing to assess large volumes of data. To determine how data quality should be defined, a literature study was conducted, which resulted in a selection of data quality dimensions deemed relevant for sensor data. The selection from the literature study contains the dimensions: Completeness, Accuracy, Timeliness, Consistency, Currency, Duplicates, and Precision. Based on this list, techniques based on data profiling was developed to assess the quality of sensor data, each technique assessed the quality regarding one specific data quality dimension. These techniques were implemented as part of an extendable framework. Each technique was evaluated by its time and space complexity, and the actual time it took for it to analyze datasets provided by Volvo Car Corporation, containing sensor data created by autonomous vehicles. Finally, approaches to improving the quality of sensor data based on the selected data quality dimensions were investigated.