Article of the Month, July 2018 (Loewenstein's lab)

July 10, 2018

DORA The Explorer: Directed Outreaching Reinforcement Action-Selection

Authors: Fox, L.,  Choshen, L. & Loewenstein, Y.

Published as a conference paper at ICLR on April 2018



Exploration is a fundamental aspect of Reinforcement Learning, typically implemented using stochastic action-selection. Exploration, however, can be more efficient if directed toward gaining new world knowledge. Visit-counters have been proven useful both in practice and in theory for directed exploration. However, a major limitation of counters is their locality. We propose a generalization of counters that can be used to evaluate the propagating exploratory value over state-action trajectories. We show that this generalization improves learning and performance over traditional counters. We also show how our method can be implemented with function approximation to efficiently learn continuous MDPs. We demonstrate this by showing that our approach surpasses state of the art performance in the Freeway Atari 2600 game.



Link: https://arxiv.org/pdf/1804.04012.pdf