Exploration Method for Reducing Uncertainty using Q-entropy in Deep Reinforcement Learning

Abstract

In this paper, we propose a novel exploration method for a Q-learning based deep reinforcement learning. The agent decides whether to explore or exploit according to the uncertainty on current state. To measure the amount of uncertainty, we utilize entropy value of action-values at each state. That is, the agent explore with random actions when the entropy of action-values is high or does greedy action when the entropy is low. Also we adopt the state visit-counter to handle the ambiguous states which means that several optimal actions exist.