Reinforcement Learning-Guided Generative Design of Surfactant Molecules - Exploring Reward Functions for Multi-Objective Molecular Optimisation

Abstract

Surfactants are chemical compounds with amphiphilic molecular structures that find application across a wide range of industries, from cleaning products and food processing to pharmaceuticals and waterproof textiles. Despite their ubiquity, many surfactants in current industrial use raise significant environmental concerns, and the physicochemical performance of even commonly used compounds often falls far short of what is theoreti cally achievable. Designing better surfactants through conventional experimental methods is slow and costly, motivating the development of computational approaches capable of navigating the vast space of possible molecular structures more intelligently. This thesis presents a computational framework for the inverse design of surfactant molecules with improved physicochemical properties, combining deep generative modelling with reinforcement learning and active learning. The framework is built on the REINVENT4 platform, employing a recurrent neural network pretrained on a large chemical database and subsequently fine-tuned on the SurfPro surfactant dataset through transfer learn ing. Reinforcement learning is then applied to guide generation toward molecules with favourable values of critical micelle concentration (pCMC) and surface tension, using surrogate models as the property oracle. Three distinct optimisation strategies are in vestigated and compared: iterative retraining, Pareto Gradient scoring, and Origin Pull scoring. The results demonstrate that the framework reliably generates chemically valid and novel molecules that extend beyond the Pareto front of the SurfPro training dataset according to surrogate model predictions. A key finding is that balanced multi-objective scoring functions both outperform single-objective weightings and better preserve chemical diver sity throughout training. However, a recurring failure mode termed here the hydrocarbon trap is identified, in which the optimisation converges on structurally degenerate molecules that lack the amphiphilic head-tail architecture required of functional surfactants. A post-processing filter based on physicochemical descriptors derived from the SurfPro dataset is introduced to screen out these spurious candidates and recover the viable subset of generated molecules. Taken together, the results constitute a proof of concept that reinforcement learning guided generative models are capable of exploring surfactant chemical space in a principled and targeted manner. While the generated candidates require experimental validation before their practical utility can be confirmed, this framework represents a promising foundation for future, more refined computational campaigns aimed at discovering next generation surfactants.