Policy Search in Infinite-Horizon Discounted Reinforcement Learning: Advances through Connections to Non-Convex Optimization: Invited Presentation

In reinforcement learning (RL), an agent moving through a state space, selects actions which cause a transition to a new state according to an unknown Markov transition density that depends on the previous state and action. After each transition, a reward that informs the quality of being in a particular state is revealed. The goal is to select the action sequence to maximize the long term accumulation of rewards, or value. We focus on the case where the policy that determines how actions are chosen is a fixed stationary distribution parameterized by a vector, the problem horizon is infinite, and the states and actions belong to continuous Euclidean subsets.