Domain Validation and Continuous Improvement of Deep Learning Models
Typ
Examensarbete för masterexamen
Master's Thesis
Master's Thesis
Program
Complex adaptive systems (MPCAS), MSc
Publicerad
2023
Författare
Darakhsh, Arash
Modellbyggare
Tidskriftstitel
ISSN
Volymtitel
Utgivare
Sammanfattning
With more industries shifting over to an industry 4.0 setting, many product manu facturing processes and related tasks are becoming more and more automated. De tection of defects in aerospace components, which is extremely crucial for aviation
safety, is normally performed by trained operators. The operators apply techniques
of non-destructive testing to acquire inspection data, which they then evaluate. In
many of these inspection tasks, digital images are a fundamental part of the inspec tion process, as in X-ray inspection of welds. For these types, efforts have been
made over the last years to increase the level of automation, thus decreasing the
burden on the operators. Deep neural network models aiding or even completely
replacing operator judgement are a main subject of interest. Before these models
can be deployed into industry they need to be thoroughly tested and validated.
In this thesis, a U-Net model is used as an example for two frequently reoccur ring problems that appear in industrial smart manufacturing. First, we investigate
whether data from one inspection task can be used to predict the relationship be tween model performance and data set size for intra-domain data sets. In this sense,
one can use information from one data set to estimate the needed sample size re quired to train a model for novel inspection tasks. Thereby, more precise estimations
about the necessary amount of data for training models in development environments
that also perform well when confronted with industrial data streams once deployed
should be possible. The second problem considers methods for continuously improv ing deployed models as more data is made available. Here full retraining, online
learning and proactive training, a recent retraining method described in Prapas et
al, Datenbank-Spektrum 21, pp. 203-212 (2021) [1], are compared. The best strat egy for continuously improving and maintaining industrial deep learning models is
researched.
The obtained results suggest that the predictive capabilities from one data set do
not carry over well to other data sets, even though belonging to the same domain
of problems. The mean average F1 error between the two considered data sets
was measured to 0.11. Despite this, alternative estimation methods, solely using
curve fitting, showed promising results. The findings from the second task favoured
standard methods of scheduled retraining for the considered U-Net, although more
extensive simulations are needed before ruling out the alternative methods consid ered in the report. In conclusion, the acquired results in this thesis support the
progress of automation of industrial inspections tasks by providing methods and
guidelines for industrial deep learning models.