Reinforcement Learning-Based Cell Balancing for Electric Vehicles

Abstract

Lithium-ion battery packs are comprised of hundreds to thousands of individual cells which, even though manufactured uniformly, exhibit small variations in their characteristics that impact their behavior during operation. These differences cause cells’ State of Charge (SOC) to become unbalanced, which can, in turn, reduce the capacity utilization efficiency of the pack [1]. Additionally, battery cells age differently over time, and fast-aged cells can cause packs with healthy cells to be retired early, without fully taking advantage of each cell. When a battery has deteriorated to around 80% of its total capacity, it is retired from electric vehicle usage [2]. To maintain batteries functioning correctly, cell SOC balancing must be done on battery packs. However, balancing the SOC of cells provides a window of opportunity to also include cells’ health into the balancing equation, aiming for the homogenization of cell aging, allowing to thoroughly utilize a battery’s resources. In this way, it is possible to both keep batteries in operating condition and potentially increase their lifespan. In this work, we develop and research a multi-cell simulation framework and Reinforcement Learning (RL) methodologies to explore the potential of cell SOC and health balancing. We propose an active balancing strategy for re-configurable cell topology with RL, in which instead of transferring energy between high SOC cells to low SOC cells, cell utilization is modulated so that the power consumption is optimally distributed based on each cell’s SOC. This strategy is applied to SOC balancing, as well as SOC and State of Health (SOH) balancing simultaneously, to potentially allow for an exhaustive utilization of the battery’s potential.