Autonomous Excavation Using Reinforcement Learning with Proximal Policy Optimization
Publicerad
Författare
Typ
Examensarbete för masterexamen
Master's Thesis
Master's Thesis
Modellbyggare
Tidskriftstitel
ISSN
Volymtitel
Utgivare
Sammanfattning
This thesis presents a reinforcement learning based approach for grading, applied in
the Volvo excavator EC550E. The work was based around a simulation model of the
excavator which facilitated easy training of the algorithm. A hydraulic controller was
trained using proximal policy optimization. Together with the hydraulic controller,
a PID was implemented as a positional controller for a complete system capable of
performing grading tasks.
Training was conducted by testing different reward functions and parameter choices
to improve policy performance. The results showcases hyperparameter evaluation,
velocity tracking accuracy for the hydraulic controller as well as grading accuracy
of the complete system. The implemented solution had an accuracy of ± 4 cm during
grading. However, the hydraulic controller was not able to consistently follow
the target velocities in the cylinders, particularly for the bucket. In future works
the hydraulic controller needs to be retrained for better precision before being deployed
in a real machine. This thesis shows the potential and possibility of replacing
traditional control policies with an machine-learning driven approach.
Beskrivning
Ämne/nyckelord
autonomous excavation, excavator, hydraulics, IMVT, machine learning, proximal policy, optimization, reinforcement learning