Artificial intelligence based marine autopilot: Trained using reinforcement learning in the Unity simulation environment

Abstract

When driving a ship, especially long distances, an autopilot is usually deployed. Since the ship is subjected to disturbances from currents and wind alike, it is not easy to keep a steady course. Furthermore, the dynamic behaviour of waves striking the vessel, known as swaying, makes the task of keeping a straight line through the sea daunting. To assist, there exist commercial autopilots. However, most of these are subject to less than simple calibration, which also is hard to keep accu rate throughout the lifespan of the boat due to wear, different load conditions, and other similar things. Also, there is generally no adaptability related to the autopilot, meaning a sudden change in engine performance will stop the autopilot from functioning. Further, the majority of today’s commercial autopilots are designed to follow a course or a heading, known as course-hold and heading-hold autopilots. Hence, there exists a desire to develop a more adaptable path-following autopilot. One way of solving the adaptability issue is to borrow the solution from the aircraft industry and use a control allocator. Given a set of global forces (usually Fx, Fy, and moment Mz) the control allocator tries to distribute these between a given set of actuators. Since the number of control signals usually is far less than the number of actuators, these systems are said to be over-actuated, and no unique solution exists. This work aims at exploring a new way of constructing computationally efficient regulation and control allocation for vessels, in the form of a path-following autopilot. The hypothesis is that, by using a Neural Network as a control allocator, better performance and adaptability than offered by present solutions can be achieved.