Title: Stochastic Adaptive Nash Certainty Equivalence Control with Population Parameter Estimation (½ hour)
Abstract: For noncooperative games the Nash Certainty Equivalence (NCE), or Mean Field (MF) methodology developed in [1,2,3,4] provides decentralized strategies which asymptotically yield Nash equilibria. The NCE (MF) control laws use only the local information of each agent on its own state evolution and knowledge of its own dynamical parameters, while the behaviour of the mass is precomputable from knowledge of the distribution of dynamical and cost function parameters throughout the mass population.
Relaxing the a priori information condition introduces the methods of parameter estimation and stochastic adaptive control (SAC) into MF control theory. A problem on this path is that where each agent needs to estimate (i) its own dynamical parameters, (ii) the distribution of the population's dynamical parameters [5], (iii) the distribution of the population's cost function parameters [6].
In the present work, each agent observes a random subset of the population of agents. Each agent estimates its own dynamical parameters via the recursive weighted least squares (RWLS) algorithm, and the distribution of the population's dynamical and cost function parameters via the maximum likelihood estimation (MLE). Under reasonable conditions, we establish: (i) the strong consistency of the self-parameter estimates and the weak convergence of the distribution functions; and that (ii) all agent systems are long run average L2 stable; (iii) the set of controls yields a (strong) e-Nash equilibrium for all e; and (iv) in the population limit the long run average cost obtained is equal to the non-adaptive long run average cost.
References:
[1] M. Huang, P. E. Caines, and R. P. Malhame, "Individual and mass behaviour
in large population stochastic wireless power control problems: Centralized
and Nash equilibrum solutions," in Proceedings of the 42nd IEEE Conference on
Decision and Control, Maui, Hawaii, December 2003, pp. 98~103.
[2] M. Huang, P. E. Caines, and R. P. Malhame, "Large-population cost-coupled
LQG problems with non-uniform agents: Individual-mass behaviour and
decentralized e-Nash equilibria," IEEE Trans. on Automatic Control, vol. 52,
no. 9, pp. 1560~1571, September 2007.
[3] T. Li and J.-F. Zhang, "Asymptotically optimal decentralized control for
large population stochastic multiagent systems," IEEE Trans. on Automatic
Control, vol. 53, no. 7, pp. 1643~1660, August 2008.
[4] P. E. Caines, "Mean Field Stochastic Control," Bode Lecture at the 48th
IEEE Conference on Decision and Control and 28th Chinese Control Conference,
Dec. 2009. [Online]. Available:
http://www.cim.mcgill.ca/~arman/Shanghai2009/Shanghai2009.zip
[5] A. C. Kizilkale and P. E. Caines, "Stochastic adaptive Nash certainty
equivalence control: Self-identification case," 2010, submitted to the 19th
International Symposium on Mathematical Theory of Networks and Systems (MTNS
2010).
[6] A. C. Kizilkale and P. E. Caines, "Stochastic adaptive Nash certainty
equivalence control: Population parameter distribution estimation," 2010,
submitted to the 49th IEEE Conference on Decision and Control, Atlanta,
Georgia.