Autonomous Navigation In Real-Time Endovascular Simulation

Abstract

This Master’s thesis investigates the use of deep reinforcement learning for autonomous navigation in a simulated endovascular environment. Specifically, a Soft Actor-Critic (SAC) algorithm is employed to train an agent to control a micro guidewire and a micro catheter for path-following tasks. The simulation environment is based on the VIST simulator, where the agent observes a 23-dimensional state representation capturing relevant path-following information. The agent was rewarded for path following, with the reward function incorporating vessel centerline alignment, penalization of deviation from the path, and progression toward the target. The agent produces continuous translation and rotation commands for the respective tools. Training was conducted on four anatomies over 1600 episodes, each consisting of up to 500 time steps depending on task completion. The best performing model emerged after 800 training episodes, which achieved a success rate of 94 % on validation data. During testing, it achieved a 96 % success rate on an unseen cerebral anatomy and an 88 % success rate on an unseen liver anatomy. This indicates that the agent learned transferable navigation strategies rather than anatomy specific memorization, which demonstrates that deep reinforcement learning is a viable approach for endovascular navigation in a simulated environment.