Implementation of Reinforcement Learning on Air Source Heat Pump Defrost Control for Full Electric Vehicles

Air source heat pumps as the heating system for full electric vehicles are drawing more and more attention in recent years. Despite the high energy efficiency, frost accumulation on the heat pump evaporator is one of the major challenges associated with air source heat pumps. The evaporator needs to be actively defrosted periodically and heat pump heating will be interrupted during defrosting process. Proper defrost control is needed to obtain high average heat pump energy efficiency. In this paper, a new method for generating air source heat pump defrost control policy using reinforcement learning is introduced. This model-free method has several advantages. It can automatically generate optimal defrost control policy instead of requiring manually determination of the control policy parameters and logics. More measurement results can be incorporated into the defrost control policy without too many changes in the reinforcement learning algorithm so that the control policy can be better optimized under wider range of working conditions. The learning features also enable the controller to adapt to the system differences and changes which are impossible to predict a priori when designing defrost control policy. The algorithm was validated using experimentally obtained heating capacity and COP data in frost growth cycle of a heat pump under different conditions. The results showed that reinforcement learning can be used to generate defrost control policy that optimizes energy consumption for various working conditions.