Reinforcement Learning With Action Discovery
Document Type
Conference Proceeding
Publication Date
5-1-2010
School
Computing Sciences and Computer Engineering
Abstract
The design of reinforcement learning solutions to many problems artificially constrain the action set available to an agent, in order to limit the exploration/sample complexity. While exploring, if an agent can discover new actions that can break through the constraints of its basic/atomic action set, then the quality of the learned decision policy could improve. On the flipside, considering all possible non-atomic actions might explode the exploration complexity. We present a potential baed solution to this dilemma, and empirically evaluate it in grid navigation tasks. In particular, we show that both the solution quality and the sample complexity improve significantly when basic reinforcement learning is coupled with action discovery. Our approach relies on reducing the number of decisions points, which is particularly suited for multiagnet coordination learning, since agents tend to learn more easily with fewer coordination problems (CPs). To demonstrate this we extend action discovery to multi-agent reinforcement learning. We show that Joint Action Learners (JALs) indeed learn coordination policies of higher quality with lower sample complexity when coupled with action discovery, in a multi-agent box-pushing task.
Publication Title
Proceedings of the AAMAS Workshop On Adaptive and Learning Agents
First Page
30
Last Page
37
Recommended Citation
Banerjee, B.,
Kraemer, L.
(2010). Reinforcement Learning With Action Discovery. Proceedings of the AAMAS Workshop On Adaptive and Learning Agents, 30-37.
Available at: https://aquila.usm.edu/fac_pubs/20626
COinS