Date of Award
Summer 7-2022
Degree Type
Masters Thesis
Degree Name
Master of Science (MS)
School
Computing Sciences and Computer Engineering
Committee Chair
Dr. Bikramjit Banerjee
Committee Chair School
Computing Sciences and Computer Engineering
Committee Member 2
Dr. Andrew Sung
Committee Member 2 School
Computing Sciences and Computer Engineering
Committee Member 3
Dr. Chaoyang Zhang
Committee Member 3 School
Computing Sciences and Computer Engineering
Abstract
Real-time strategy (RTS) games have provided a fertile ground for AI research with notable recent successes based on deep reinforcement learning (RL). However, RL remains a data-hungry approach featuring a high sample complexity. In this thesis, we focus on a sample complexity reduction technique called reinforcement learning as a rehearsal (RLaR), and on the RTS game of MicroRTS to formulate and evaluate it. RLaR has been formulated in the context of action-value function based RL before. Here we formulate it for a different RL framework, called actor-critic RL. We show that on the one hand the actor-critic framework allows RLaR to be much simpler, but on the other hand it leaves room for a key component of RLaR--a prediction function that relates a learner's observations with that of its opponent. This function, when leveraged for exploration, accelerates RL as our experiments in MicroRTS show. Further experiments provide evidence that RLaR may reduce actor noise compared to a variant that does not utilize RLaR's exploration. This study provides the first evaluation of RLaR's efficacy in a domain with a large strategy space.
Recommended Citation
Manandhar, Shiron, "Reinforcement Actor-Critic Learning As A Rehearsal In MicroRTS" (2022). Master's Theses. 914.
https://aquila.usm.edu/masters_theses/914