Reinforcement Learning In Many-Agent Settings Under Partial Observability
Document Type
Conference Proceeding
Publication Date
8-1-2022
School
Computing Sciences and Computer Engineering
Abstract
Recent renewed interest in multi-agent reinforcement learning (MARL) has generated an impressive array of techniques that leverage deep RL, primarily actor-critic architectures, and can be applied to a limited range of settings in terms of observability and communication. However, a continuing limitation of much of this work is the curse of dimensionality when it comes to representations based on joint actions, which grow exponentially with the number of agents. In this paper, we squarely focus on this challenge of scalability. We apply the key insight of action anonymity to a recently presented actor-critic based MARL algorithm, interactive A2C. We introduce a Dirichlet-multinomial model for maintaining beliefs over the agent population when agents’ actions are not perfectly observable. We show that the posterior is a mixture of Dirichlet distributions that we approximate as a single component for tractability. We also show that the prediction accuracy of this method increases with more agents. Finally we show empirically that our method can learn optimal behaviors in two recently introduced pragmatic domains with large agent population, and demonstrates robustness in partially observable environments.
Publication Title
Proceedings of the 38th Conference On Uncertainty In Artificial Intelligence
First Page
780
Last Page
789
Recommended Citation
He, K.,
Doshi, P.,
Banerjee, B.
(2022). Reinforcement Learning In Many-Agent Settings Under Partial Observability. Proceedings of the 38th Conference On Uncertainty In Artificial Intelligence, 780-789.
Available at: https://aquila.usm.edu/fac_pubs/20613
COinS