Policy Learning and Off-Policy Evaluation with Application to the Sequential Treatment of Rheumatoid Arthritis

Abstract

This thesis addresses the challenge of optimizing treatment strategies for patients with Rheumatoid Arthritis (RA) after switch to second-line biologic or targeted synthetic disease-modifying antirheumatic drugs (b/tsDMARDs) therapy. Given the complexity of RA treatment and the limitations of randomized controlled trials (RCTs) in real-world settings, we leverage historical data from the CorEvitas RA registry to model and evaluate alternative treatment strategies. We employ interpretable machine learning models to capture existing treatment decision patterns and propose a target policy tailored to individual patient needs. Our approach integrates features derived from patient history, treatment patterns, and clinical characteristics to enhance prediction accuracy. We introduce a novel two-stage combined model that first predicts whether a patient will switch therapies and then determines the most appropriate subsequent therapy. The proposed target policy is compared against current guidelines, such as those provided by the European Alliance of Associations for Rheumatology (EULAR), to identify potential improvements. Finally, we validate the proposed policy through off-policy evaluation, utilizing techniques such as importance sampling and weighted importance sampling to assess its effectiveness in real-world scenarios. The findings suggest that the proposed approach can lead to more personalized and effective treatment plans, potentially improving patient outcomes in RA management.