Deep learning for semantic segmentation of FIB-SEM volumetric image data
Typ
Examensarbete för masterexamen
Program
Publicerad
2020
Författare
Lennerfors, Mikael
Johansson Visuri, William
Modellbyggare
Tidskriftstitel
ISSN
Volymtitel
Utgivare
Sammanfattning
Focused ion beam scanning electron microscopy (FIB-SEM) is a well-established
microscopy technique for 3D imaging of porous materials. Porous materials, such as
ethyl cellulose (EC) and hydroxypropyl cellulose (HPC) polymer blends, are used as
coating materials on pharmaceutical tablets or pellets to control drug release. These
porous materials form a network of pores that allow the drug release. Therefore,
these formed microstructures are essential to optimize the design of the coating materials.
These structures can be analyzed by semantic segmentation of the porous
network, thereby separating the pores from the solid parts of the material. However,
an overlap of grayscale intensities between solid regions and porous regions occurs.
This thesis explores different convolutional neural networks (CNN) for the segmentation
of FIB-SEM 2D image data of porous materials. All four investigated CNNs in
this thesis are based on the so-called U-Net architecture developed by Ronneberg et
al., in 2015. Every network reads a 2D grayscale image which it tries to segment between
pores and solid material. The output is a 2D mask where the white regions are
pores, and black regions are solid background. The FIB-SEM data suffers from an
imbalance of roughly 30% pores and 70% solid background. This imbalance makes
accuracy a poor metric for evaluation segmentation performance, since classifying
everything as pores leads to 70% accuracy. Instead, a metric called Intersection over
Union (IoU), or Jaccard index, was implemented. IoU has been shown to tackle an
imbalanced problem better. The chosen metric will also impact which loss function
to use when training the network. We found that implementing IoU in a mixed
loss function with weighted Binary Cross-Entropy (wBCE) worked better than just
binary cross-entropy, which is a standard loss function. The best performing architecture,
in terms of both IoU score and predicted porosity was MultiResU-Net.
MultiResU-Net had a test IoU of 70.5% and a predicted porosity of 30.3%, which
is a good agreement in terms of porosity. Every network architecture that was investigated
managed to perform reasonable segmentation of each image. Training of
the network and optimization of the hyperparameter were computationally heavy.
Due to time restriction, we were not able to optimize the parameter settings fully.
To enable the neural networks full potential, cloud-computing could be used in the
future.
Beskrivning
Ämne/nyckelord
Deep learning, convolutional neural network, data augmentation, U-Net, Jaccard index, Focused ion beam scanning electron microscopy.