Predicting antibiotic resistance using fusion transformers

Abstract

Antimicrobial resistance threatens recent gains in global public health by making it more difficult to treat infections. Clinicians must administer treatments based on limited diagnostic information and increasing resistance complicates these decisions. This thesis project explores ways to support this process by developing a framework for training a transformer model using data fusion of patient and genotype data with phenotype data to make individualized predictions of antibiotic resistance in Escherichia coli on these multimodal data. To achieve this, the model was trained in two stages: first, the model was pre-trained on large volumes of unimodal data using masked language modeling to learn patterns within the modalities; and second, the model was fine-tuned on a small multimodal dataset to learn patterns across modalities. To evaluate pre-training strategies, the model was fine-tuned on two clinically relevant tasks and smaller training sets. To determine the value of introducing multimodality and the effect of genotype data availability on performance, the model was fine-tuned on varying levels of available genotype information. The results show that the model performs well on the fine-tuning tasks, that pretraining on unimodal data improves performance, and that the model can extrapolate well from small training sets and incomplete data. Therefore, it can be concluded that this work has achieved the aim of developing a model that can make accurate predictions based on limited diagnostic information. Importantly, large performance improvements were observed with increasing genotype data availability, especially on difficult antibiotics. Furthermore, the model was better able to utilize available genotype information when pre-trained. However, while no clear conclusion on the best pre-training strategy can be drawn from the results of this work, they indicate that using systematic class masking in pre-training yields the highest performance. Future research should further investigate the best strategy for pre-training the model, how the model utilizes genotype data to improve performance, and how genotype data affects performance on limited training data.