Developing a one-to-many generation LLM for diverse, accurate and efficient retrosynthesis
Ladda ner
Typ
Examensarbete för masterexamen
Master's Thesis
Master's Thesis
Program
Computer systems and networks (MPCSN), MSc
Publicerad
2024
Författare
Li, Junyong
Modellbyggare
Tidskriftstitel
ISSN
Volymtitel
Utgivare
Sammanfattning
One of the most common applications of deep learning for cheminformatics is retrosynthesis,
which is a task of predicting reactants given a chemical product. After
transformer was invented, it has been widely used for retrosynthesis. Chemformer
is a transformer-based model, which was pre-trained using SMILES of chemical
molecules first and can be fine-tuned for retrosynthesis. The model achieves stateof-
the-art performance on this task. Retrosynthesis task expects multiple predictions
of reactants. Chemformer uses beam search or multinomial search to get multiple
predictions, which results in a lack of diversity, accuracy and efficiency of the model.
In this project, the sphere projection strategy, which is a one-to-many generation
strategy, was applied to Chemformer to enable it to generate multiple predictions.
The sphere projection achieves one-to-many generation by introducing variations of
source embedding of encoder and combining those variations with a single-prediction
sampler, such as greedy search and multinomial search (multinomial size = 1). By
comparing the modified Chemformer with sphere projection strategy to the baseline
Chemformer, it was shown that the strategy can improve diversity, accuracy and
efficiency by 197%, 7% and 4% respectively for beam search, and 101%, 2% and
17% respectively for multinomial search.
Beskrivning
Ämne/nyckelord
Retrosynthesis , LLM , large-language model , one-to-many generation , machine learning , deep learning , transformer , diversity, accuracy , efficiency