Stochastic games for the smart grid energy management with prospect prosumers

Seyed Rasoul Etesami, Walid Saad, Narayan B. Mandayam, H. Vincent Poor

Research output: Contribution to journalArticle

Abstract

In this paper, the problem of the smart grid energy management under stochastic dynamics is investigated. In the considered model, at the demand side, it is assumed that customers can act as prosumers who own renewable energy sources and can both produce and consume energy. Due to the coupling between the prosumers' decisions and the stochastic nature of renewable energy, the interaction among prosumers is formulated as a stochastic game, in which each prosumer seeks to maximize its payoff, in terms of revenues, by controlling its energy consumption and demand. In particular, the subjective behavior of prosumers is explicitly reflected into their payoff functions using the prospect theory, a powerful framework that allows modeling real-life human choices, rather than objective, user-agnostic decisions, as normative models do. For this prospect-based stochastic game, it is shown that there always exists a stationary Nash equilibrium where the prosumers' trading policies in the equilibrium are independent of the time and their histories of the play. Moreover, to obtain one of such equilibrium policies, a novel distributed algorithm with no information sharing among prosumers is proposed and shown to converge to an \epsilon-Nash equilibrium in which each prosumer is able to achieve its optimal payoff in an equilibrium up to a small additive error \epsilon. On the other hand, at the supply side, the interaction between the utility company and the prosumers is formulated as an online optimization problem in which the utility company's goal is to learn its optimal energy allocation rules. For this case, it is shown that such an optimization problem admits a no-regret algorithm meaning that regardless of the actual outcome of the game among the prosumers, the utility company can follow a strategy that mitigates its allocation costs as if it knew the entire demand market a priori. Simulation results justify the convergence of the proposed algorithms and present new insights toward more efficient energy management in the smart grids.

Original languageEnglish (US)
Pages (from-to)2327-2342
Number of pages16
JournalIEEE Transactions on Automatic Control
Volume63
Issue number8
DOIs
StatePublished - Aug 2018

Fingerprint

Energy management
Industry
Parallel algorithms
Energy utilization
Costs

Keywords

  • Distributed learning
  • energy management
  • prospect theory
  • smart grid
  • stationary Nash equilibrium
  • stochastic game

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Computer Science Applications
  • Electrical and Electronic Engineering

Cite this

Stochastic games for the smart grid energy management with prospect prosumers. / Etesami, Seyed Rasoul; Saad, Walid; Mandayam, Narayan B.; Poor, H. Vincent.

In: IEEE Transactions on Automatic Control, Vol. 63, No. 8, 08.2018, p. 2327-2342.

Research output: Contribution to journalArticle

Etesami, Seyed Rasoul ; Saad, Walid ; Mandayam, Narayan B. ; Poor, H. Vincent. / Stochastic games for the smart grid energy management with prospect prosumers. In: IEEE Transactions on Automatic Control. 2018 ; Vol. 63, No. 8. pp. 2327-2342.
@article{a431ad2e2c6e49cd82f3521e479e719e,
title = "Stochastic games for the smart grid energy management with prospect prosumers",
abstract = "In this paper, the problem of the smart grid energy management under stochastic dynamics is investigated. In the considered model, at the demand side, it is assumed that customers can act as prosumers who own renewable energy sources and can both produce and consume energy. Due to the coupling between the prosumers' decisions and the stochastic nature of renewable energy, the interaction among prosumers is formulated as a stochastic game, in which each prosumer seeks to maximize its payoff, in terms of revenues, by controlling its energy consumption and demand. In particular, the subjective behavior of prosumers is explicitly reflected into their payoff functions using the prospect theory, a powerful framework that allows modeling real-life human choices, rather than objective, user-agnostic decisions, as normative models do. For this prospect-based stochastic game, it is shown that there always exists a stationary Nash equilibrium where the prosumers' trading policies in the equilibrium are independent of the time and their histories of the play. Moreover, to obtain one of such equilibrium policies, a novel distributed algorithm with no information sharing among prosumers is proposed and shown to converge to an \epsilon-Nash equilibrium in which each prosumer is able to achieve its optimal payoff in an equilibrium up to a small additive error \epsilon. On the other hand, at the supply side, the interaction between the utility company and the prosumers is formulated as an online optimization problem in which the utility company's goal is to learn its optimal energy allocation rules. For this case, it is shown that such an optimization problem admits a no-regret algorithm meaning that regardless of the actual outcome of the game among the prosumers, the utility company can follow a strategy that mitigates its allocation costs as if it knew the entire demand market a priori. Simulation results justify the convergence of the proposed algorithms and present new insights toward more efficient energy management in the smart grids.",
keywords = "Distributed learning, energy management, prospect theory, smart grid, stationary Nash equilibrium, stochastic game",
author = "Etesami, {Seyed Rasoul} and Walid Saad and Mandayam, {Narayan B.} and Poor, {H. Vincent}",
year = "2018",
month = "8",
doi = "10.1109/TAC.2018.2797217",
language = "English (US)",
volume = "63",
pages = "2327--2342",
journal = "IEEE Transactions on Automatic Control",
issn = "0018-9286",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
number = "8",

}

TY - JOUR

T1 - Stochastic games for the smart grid energy management with prospect prosumers

AU - Etesami, Seyed Rasoul

AU - Saad, Walid

AU - Mandayam, Narayan B.

AU - Poor, H. Vincent

PY - 2018/8

Y1 - 2018/8

N2 - In this paper, the problem of the smart grid energy management under stochastic dynamics is investigated. In the considered model, at the demand side, it is assumed that customers can act as prosumers who own renewable energy sources and can both produce and consume energy. Due to the coupling between the prosumers' decisions and the stochastic nature of renewable energy, the interaction among prosumers is formulated as a stochastic game, in which each prosumer seeks to maximize its payoff, in terms of revenues, by controlling its energy consumption and demand. In particular, the subjective behavior of prosumers is explicitly reflected into their payoff functions using the prospect theory, a powerful framework that allows modeling real-life human choices, rather than objective, user-agnostic decisions, as normative models do. For this prospect-based stochastic game, it is shown that there always exists a stationary Nash equilibrium where the prosumers' trading policies in the equilibrium are independent of the time and their histories of the play. Moreover, to obtain one of such equilibrium policies, a novel distributed algorithm with no information sharing among prosumers is proposed and shown to converge to an \epsilon-Nash equilibrium in which each prosumer is able to achieve its optimal payoff in an equilibrium up to a small additive error \epsilon. On the other hand, at the supply side, the interaction between the utility company and the prosumers is formulated as an online optimization problem in which the utility company's goal is to learn its optimal energy allocation rules. For this case, it is shown that such an optimization problem admits a no-regret algorithm meaning that regardless of the actual outcome of the game among the prosumers, the utility company can follow a strategy that mitigates its allocation costs as if it knew the entire demand market a priori. Simulation results justify the convergence of the proposed algorithms and present new insights toward more efficient energy management in the smart grids.

AB - In this paper, the problem of the smart grid energy management under stochastic dynamics is investigated. In the considered model, at the demand side, it is assumed that customers can act as prosumers who own renewable energy sources and can both produce and consume energy. Due to the coupling between the prosumers' decisions and the stochastic nature of renewable energy, the interaction among prosumers is formulated as a stochastic game, in which each prosumer seeks to maximize its payoff, in terms of revenues, by controlling its energy consumption and demand. In particular, the subjective behavior of prosumers is explicitly reflected into their payoff functions using the prospect theory, a powerful framework that allows modeling real-life human choices, rather than objective, user-agnostic decisions, as normative models do. For this prospect-based stochastic game, it is shown that there always exists a stationary Nash equilibrium where the prosumers' trading policies in the equilibrium are independent of the time and their histories of the play. Moreover, to obtain one of such equilibrium policies, a novel distributed algorithm with no information sharing among prosumers is proposed and shown to converge to an \epsilon-Nash equilibrium in which each prosumer is able to achieve its optimal payoff in an equilibrium up to a small additive error \epsilon. On the other hand, at the supply side, the interaction between the utility company and the prosumers is formulated as an online optimization problem in which the utility company's goal is to learn its optimal energy allocation rules. For this case, it is shown that such an optimization problem admits a no-regret algorithm meaning that regardless of the actual outcome of the game among the prosumers, the utility company can follow a strategy that mitigates its allocation costs as if it knew the entire demand market a priori. Simulation results justify the convergence of the proposed algorithms and present new insights toward more efficient energy management in the smart grids.

KW - Distributed learning

KW - energy management

KW - prospect theory

KW - smart grid

KW - stationary Nash equilibrium

KW - stochastic game

UR - http://www.scopus.com/inward/record.url?scp=85040985844&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85040985844&partnerID=8YFLogxK

U2 - 10.1109/TAC.2018.2797217

DO - 10.1109/TAC.2018.2797217

M3 - Article

AN - SCOPUS:85040985844

VL - 63

SP - 2327

EP - 2342

JO - IEEE Transactions on Automatic Control

JF - IEEE Transactions on Automatic Control

SN - 0018-9286

IS - 8

ER -