Sequential decomposition and policy iteration schemes for M-player games with partial weak coupling

We formulate two general classes of M-player deterministic and stochastic nonzero-sum games where the players can be placed into two groups such that there are strong interactions within each group and a weak interaction between the two groups. This weak interaction is characterized in terms of a small parameter ε which, when set equal to zero, leads to two independent nonzero-sum games. Under the Nash equilibrium solution concept both within and in between the groups, we study the merits of an iterative method for the construction of the equilibrium by solving simpler problems at each stage of the iteration. In this iterative scheme, the zeroth order solution is the Nash equilibrium of the two independent games obtained by setting ε = 0, whereas the higher-order solutions are Nash equilibria of quadratic games, even though the original problem may have nonquadratic cost functions.