Autonomous Excavation Using Reinforcement Learning with Proximal Policy Optimization

Publicerad

Typ

Examensarbete för masterexamen
Master's Thesis

Modellbyggare

Tidskriftstitel

ISSN

Volymtitel

Utgivare

Sammanfattning

This thesis presents a reinforcement learning based approach for grading, applied in the Volvo excavator EC550E. The work was based around a simulation model of the excavator which facilitated easy training of the algorithm. A hydraulic controller was trained using proximal policy optimization. Together with the hydraulic controller, a PID was implemented as a positional controller for a complete system capable of performing grading tasks. Training was conducted by testing different reward functions and parameter choices to improve policy performance. The results showcases hyperparameter evaluation, velocity tracking accuracy for the hydraulic controller as well as grading accuracy of the complete system. The implemented solution had an accuracy of ± 4 cm during grading. However, the hydraulic controller was not able to consistently follow the target velocities in the cylinders, particularly for the bucket. In future works the hydraulic controller needs to be retrained for better precision before being deployed in a real machine. This thesis shows the potential and possibility of replacing traditional control policies with an machine-learning driven approach.

Beskrivning

Ämne/nyckelord

autonomous excavation, excavator, hydraulics, IMVT, machine learning, proximal policy, optimization, reinforcement learning

Citation

Arkitekt (konstruktör)

Geografisk plats

Byggnad (typ)

Byggår

Modelltyp

Skala

Teknik / material

Index

item.page.endorsement

item.page.review

item.page.supplemented

item.page.referenced