Minimizing search time for finding an effective treatment: Learning a near-optimal policy using constrained algorithms, approximations, and causal inference

Abstract

Patients sometimes have to try several treatments before the one that best alleviates their symptoms is found. Since each trial of an unsuccessful treatment can be both costly and prolong patient suffering, making this search as efficient as possible is of great importance. We have developed a solution in two parts. (i) A constraint that balances the need to find a better treatment versus the desire to minimize the number of treatments tried. (ii) A dynamic programming algorithm and a greedy algorithm that uses the constraint for finding a policy that finds a good treatment in as few trials as possible. We also develop different methods of estimating potential outcomes and computing the constraint. The algorithms are trained on observational data using causal inference to learn a policy based on true causal effects. The novel algorithms are then evaluated and compared to baseline algorithms on synthetic and real-world antibiotic resistance data.