Domain Validation and Continuous Improvement of Deep Learning Models
Examensarbete för masterexamen
Complex adaptive systems (MPCAS), MSc
With more industries shifting over to an industry 4.0 setting, many product manu facturing processes and related tasks are becoming more and more automated. De tection of defects in aerospace components, which is extremely crucial for aviation safety, is normally performed by trained operators. The operators apply techniques of non-destructive testing to acquire inspection data, which they then evaluate. In many of these inspection tasks, digital images are a fundamental part of the inspec tion process, as in X-ray inspection of welds. For these types, efforts have been made over the last years to increase the level of automation, thus decreasing the burden on the operators. Deep neural network models aiding or even completely replacing operator judgement are a main subject of interest. Before these models can be deployed into industry they need to be thoroughly tested and validated. In this thesis, a U-Net model is used as an example for two frequently reoccur ring problems that appear in industrial smart manufacturing. First, we investigate whether data from one inspection task can be used to predict the relationship be tween model performance and data set size for intra-domain data sets. In this sense, one can use information from one data set to estimate the needed sample size re quired to train a model for novel inspection tasks. Thereby, more precise estimations about the necessary amount of data for training models in development environments that also perform well when confronted with industrial data streams once deployed should be possible. The second problem considers methods for continuously improv ing deployed models as more data is made available. Here full retraining, online learning and proactive training, a recent retraining method described in Prapas et al, Datenbank-Spektrum 21, pp. 203-212 (2021) , are compared. The best strat egy for continuously improving and maintaining industrial deep learning models is researched. The obtained results suggest that the predictive capabilities from one data set do not carry over well to other data sets, even though belonging to the same domain of problems. The mean average F1 error between the two considered data sets was measured to 0.11. Despite this, alternative estimation methods, solely using curve fitting, showed promising results. The findings from the second task favoured standard methods of scheduled retraining for the considered U-Net, although more extensive simulations are needed before ruling out the alternative methods consid ered in the report. In conclusion, the acquired results in this thesis support the progress of automation of industrial inspections tasks by providing methods and guidelines for industrial deep learning models.