KNeeRF: Novel View Synthesis of a Knee's Interior - A 3D Modeling Approach to Optimize Graft Placement

Abstract

Abstract Anterior cruciate ligament (ACL) reconstruction surgeries suffer from high relapse rates, due to the hard problem of placing the replacement graft. 3D models of knees could be used as training material for medical professionals to practice placing the ligaments to reduce relapse rates. We have developed a technique that can produce realistic renders of knees’ interiors that can further be used in this context. Our technique is based on Neural Radiance Fields (NeRF) to create novel views of the knee, where we compared the original NeRF implementation and two other implementations, Self-Calibrating NeRF (SCNeRF) and Nerf in the Wild (NeRF-W) to choose a starting point for further development. We landed on using NeRF-W as our baseline model. We have made several extensions to NeRF-W to further improve the application to ACL reconstruction surgeries, such as correcting ray distortions to produce accurate renderings and using segmentation masks to help the model remove medical tools from the renderings. We apply our system to data sets of ACL reconstruction surgeries and demonstrate results that surpass those of the models compared. We present an average improvement compared to NeRF-W with 5.7% in PSNR, 0.69% in SSIM, and 23.5% in LPIPS.