Human-in-the-loop control of molecular reinforcement learning with online adaptive classifiers

Abstract

The early stage of drug discovery faces significant challenges of screening through a vast number of compounds to identify potential drug candidates for specific diseases. Amidst a range of AI-based systems employed in efficiently identifying or generating potential drug candidates, this thesis focuses on REINVENT, a prominent production-ready tool for de novo design. Despite being advanced with multiple scoring options, it is challenging for REINVENT to capture human intuitions for generating desired outcomes. This thesis explores the significance of integrating human feedback to REINVENT through interactive visualization and online learning models. A range of methods have been employed during the development, First to enhance users’ understanding of generated compounds, diverse compound generation was studied, leading to an interactive visualization platform. We aim to offer a platform enabling effective user guidance. Second, to capture human preference, human feedback was integrated as a separate scoring function using online learning models. Considering the time and resources, surrogate user models were employed to represent real chemists, allowing for efficient development. During this testing, various aspects of the proposed system, including different online learning models, rating frequencies, sampling methods, and the number of rated molecules were tested and estimated. An evaluation experiment involving eight human participants demonstrated that integrating the HITL system to REINVENT can accelerate the drug discovery process by integrating AI capabilities with human expertise. It can effectively enhance the identification of valuable molecules, reduces compound analysis time, and ultimately results in improved patient outcomes and cost-effectiveness.