Decision-Making Problem for Two-Player Markov Game: Perspective of Feedback Control

This paper investigates the decision-making problem for two-player Markov game from the perspective of feedback control, and we hope to find solutions which are explicitly given. For the noncooperative game, we firstly prove the existence and uniqueness of Nash equilibrium pair. Then based on the nonlinear dynamic equation of Markov chain and the quadratic performance metrics, we deduce the theoretical solution via dynamic programming. Further, taking into account restrictions on the transition probabilities, practical solution is then given by comparing the location of theoretical solution with the admissible domain. Finally, an iterative algorithm is proposed to search for the Nash equilibrium pair. Following the similar steps, a theoretical solution is deduced for a cooperative Markov game. By using the Lagrangian method, we obtain the practical solution with the corresponding algorithm given. Numerical simulations verify the effectiveness of our proposed method.