Deep learning for semantic segmentation of FIB-SEM volumetric image data

Abstract

Focused ion beam scanning electron microscopy (FIB-SEM) is a well-established microscopy technique for 3D imaging of porous materials. Porous materials, such as ethyl cellulose (EC) and hydroxypropyl cellulose (HPC) polymer blends, are used as coating materials on pharmaceutical tablets or pellets to control drug release. These porous materials form a network of pores that allow the drug release. Therefore, these formed microstructures are essential to optimize the design of the coating materials. These structures can be analyzed by semantic segmentation of the porous network, thereby separating the pores from the solid parts of the material. However, an overlap of grayscale intensities between solid regions and porous regions occurs. This thesis explores different convolutional neural networks (CNN) for the segmentation of FIB-SEM 2D image data of porous materials. All four investigated CNNs in this thesis are based on the so-called U-Net architecture developed by Ronneberg et al., in 2015. Every network reads a 2D grayscale image which it tries to segment between pores and solid material. The output is a 2D mask where the white regions are pores, and black regions are solid background. The FIB-SEM data suffers from an imbalance of roughly 30% pores and 70% solid background. This imbalance makes accuracy a poor metric for evaluation segmentation performance, since classifying everything as pores leads to 70% accuracy. Instead, a metric called Intersection over Union (IoU), or Jaccard index, was implemented. IoU has been shown to tackle an imbalanced problem better. The chosen metric will also impact which loss function to use when training the network. We found that implementing IoU in a mixed loss function with weighted Binary Cross-Entropy (wBCE) worked better than just binary cross-entropy, which is a standard loss function. The best performing architecture, in terms of both IoU score and predicted porosity was MultiResU-Net. MultiResU-Net had a test IoU of 70.5% and a predicted porosity of 30.3%, which is a good agreement in terms of porosity. Every network architecture that was investigated managed to perform reasonable segmentation of each image. Training of the network and optimization of the hyperparameter were computationally heavy. Due to time restriction, we were not able to optimize the parameter settings fully. To enable the neural networks full potential, cloud-computing could be used in the future.