Distributional Deep Reinforcement Learning with a Mixture of Gaussians
- Authors
- Choi, Yunho; Lee, Kyungjae; Oh, Songhwai
- Issue Date
- May-2019
- Publisher
- IEEE
- Citation
- 2019 INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION (ICRA), pp 9791 - 9797
- Pages
- 7
- Journal Title
- 2019 INTERNATIONAL CONFERENCE ON ROBOTICS AND AUTOMATION (ICRA)
- Start Page
- 9791
- End Page
- 9797
- URI
- https://scholarworks.bwise.kr/cau/handle/2019.sw.cau/59369
- DOI
- 10.1109/ICRA.2019.8793505
- ISSN
- 1050-4729
2577-087X
- Abstract
- In this paper, we propose a novel distributional reinforcement learning (RL) method which models the distribution of the sum of rewards using a mixture density network. Recently, it has been shown that modeling the randomness of the return distribution leads to better performance in Atari games and control tasks. Despite the success of the prior work, it has limitations which come from the use of a discrete distribution. First, it needs a projection step and softmax parametrization for the distribution, since it minimizes the KL divergence loss. Secondly, its performance depends on discretization hyperparameters such as the number of atoms and bounds of the support which require domain knowledge. We mitigate these problems with the proposed parameterization, a mixture of Gaussians. Furthermore, we propose a new distance metric called the Jensen-Tsallis distance, which allows the computation of the distance between two mixtures of Gaussians in a closed form. We have conducted various experiments to validate the proposed method, including Atari games and autonomous vehicle driving.
- Files in This Item
- There are no files associated with this item.
- Appears in
Collections - College of Software > Department of Artificial Intelligence > 1. Journal Articles
![qrcode](https://api.qrserver.com/v1/create-qr-code/?size=55x55&data=https://scholarworks.bwise.kr/cau/handle/2019.sw.cau/59369)
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.