We address the problem of voltage control in power distribution networks by coordinating the distributed energy resources (DERs) at different buses. This problem has been investigated actively via either distributed optimization-based or local feedback control-based approaches. The former one requires a strongly-connected communication network among all DERs for implementing the optimization algorithms, which is not yet realistic in existing distribution systems with under-deployed communication infrastructure. The latter one, on the other hand, has been proven to suffer from loss of network-wide operational optimality. In this paper, we propose a game-theoretic characterization for semi-local voltage control with only a locally connected communication network. We analyze the existence and uniqueness of the generalized Nash equilibrium (GNE) for this characterization, and develop a fully distributed equilibrium-learning algorithm that hinges on only neighbor-to-neighbor information exchange of DERs. Provable convergence results are provided along with numerical tests, to illustrate the robust convergence property of our algorithm.