Humanoid Balance and Step Recovery in Public Transport Scenarios using Reinforcement Learning

Abstract

Humanoid robots that travel by public transport must remain standing while the vehicle floor accelerates, brakes, and turns beneath them. This thesis develops reinforcement-learning policies that keep a 29-degree-of-freedom Unitree G1 balanced on a finite moving floor, using only the robot’s own inertial and joint measurements at deployment. The policies act directly on joint-position targets and are trained with proximal policy optimization and an asymmetric critic in Isaac Lab. Two support conditions form the main study. Without a handle, event-gated rewards allow the robot to step when its current stance can no longer arrest the motion. With a rigid handle, the reward keeps the feet as the primary support, limits the load on the hand, and returns the robot to a support-ready stance after a disturbance. A ceiling-pivoted suspended handle is studied in simulation as an extension. In simulation, the no-handle policy completes almost all straight-line but only about half of the turning episodes of its training distribution, and it recovers by stepping with the disturbance. In a straight-line severity sweep, it leaves the usable floor area from a peak acceleration of about 2 m/s2, whereas the rigid-handle policy does not step and exceeds the 10 N m hand-moment limit of its training from about 4 m/s2. A sweep of the exported policies in MuJoCo shows the same limits. In a moving research van on a closed proving ground, both policies were compared with the manufacturer’s standing controller. Across five paired straight-road maneuvers, the no-handle policy reduced the median torso-attitude variation from 6.6◦ to 1.9◦ and the median whole-body state deviation from 4.7 to 1.6; in turning, it reduced the attitude variation from 3.1◦ to 1.4◦ at the cost of more whole-body motion. The default controller lost balance in S4 and S5 (target speeds of 36 and 45 km/h), leaving 3/5 trials (60%) without tether intervention, compared with 5/5 (100%) for each learned configuration. The rigid-handle configuration produced the smallest median whole-body state deviation in both straight driving (0.039 versus 4.7 for the default condition) and turning (0.131 versus 0.95), with median torso-attitude variations of 1.3◦ and 1.7◦, respectively. The results show that joint-level policies trained on a moving platform transfer to a real vehicle, and that the available support area or the admissible hand load, rather than upright balance itself, sets the limit of the tolerable disturbance.