Maximum Causal Tsallis Entropy Imitation Learning
- Authors
- Lee, Kyungjae; Choi, Sungjoon; Oh, Songhwai
- Issue Date
- May-2018
- Publisher
- NEURAL INFORMATION PROCESSING SYSTEMS (NIPS)
- Citation
- ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 31 (NIPS 2018), v.31
- Journal Title
- ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 31 (NIPS 2018)
- Volume
- 31
- URI
- https://scholarworks.bwise.kr/cau/handle/2019.sw.cau/59373
- ISSN
- 1049-5258
- Abstract
- In this paper, we propose a novel maximum causal Tsallis entropy (MCTE) framework for imitation learning which can efficiently learn a sparse multi-modal policy distribution from demonstrations. We provide the full mathematical analysis of the proposed framework. First, the optimal solution of an MCTE problem is shown to be a sparsemax distribution, whose supporting set can be adjusted. The proposed method has advantages over a softmax distribution in that it can exclude unnecessary actions by assigning zero probability. Second, we prove that an MCTE problem is equivalent to robust Bayes estimation in the sense of the Brier score. Third, we propose a maximum causal Tsallis entropy imitation learning (MCTEIL) algorithm with a sparse mixture density network (sparse MDN) by modeling mixture weights using a sparsemax distribution. In particular, we show that the causal Tsallis entropy of an MDN encourages exploration and efficient mixture utilization while Shannon entropy is less effective.
- Files in This Item
- There are no files associated with this item.
- Appears in
Collections - College of Software > Department of Artificial Intelligence > 1. Journal Articles
![qrcode](https://api.qrserver.com/v1/create-qr-code/?size=55x55&data=https://scholarworks.bwise.kr/cau/handle/2019.sw.cau/59373)
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.