Autonomous Drug Design with Reinforcement Learning

Abstract

The drug design process is currently one of manual trial and error, where potential drug candidates are proposed by chemists, synthesized in laboratories, and then tested and analyzed for properties and efficacy. This process, also called the Design- Make-Test-Analyze (DMTA) cycle, is repeated until a satisfying drug candidate is reached. Statistical models to sample the chemical space and generate potential molecules, combined with automated laboratories and machine learning allows for the automatization of the DMTA-cycle. However, there is still a need for improvement and this is where our project comes in.

One way to improve the automatization of the DMTA-cycle is to reduce the number of cycles needed, and our aim was to achieve this by improving the selection of compounds. To do this, we developed two deep reinforcement learning algorithms, Deep-Q Network (DQN) and Double Deep-Q Network (DDQN), and compared these to two baseline selection algorithms. This approach was chosen as it translates well into the drug development field. Reinforcement learning in drug discovery works by exploring the proposed molecules to find potential candidates and selecting the most promising ones based on molecular similarity to some predetermined properties.

Ultimately, the project was unsuccessful. The baseline selection algorithms using random and greedy selection approaches proved more efficient and accurate than the two algorithms we developed. The involvement of reinforcement learning agents when selecting compounds seemed to cloud the generative model’s understanding of what constitutes a good molecule, and thereby reduced the quality of proposed molecules for both the implemented selection algorithms. However, we found that the DQN algorithm shows some signs of promise and can, with some fine-tuning, potentially be brought up to par with the baseline selection algorithms, and perhaps even surpass them.