Generalized Prioritized Sweeping
D. Andre, N. Friedman, and R. Parr
To appear, NIPS 1997.
A preliminary version appears in
Reinforcment Learning Workshop at the Fourteenth International
Conference on Machine Learning. 1997.
Postscript version
(120K)
PDF version.
Abstract
Prioritized sweeping is a model-based reinforcement
learning method that attempt to focus the agent's limited
computational resources to achieve a good estimate of the value of
environment states. The classic account of prioritized sweeping uses
an explicit, state-based representation of the value, reward, and model
parameters.
Such a representation is unwieldy for dealing with complex
environments and there is growing interest in learning with more
compact representations. We claim that classic prioritized sweeping is
ill-suited for learning with such representations. To overcome this
deficiency, we introduce generalized prioritized sweeping, a
principled method for generating representation-specific algorithms
for model-based reinforcement learning. We then apply this method for
several representations, including state-based models and generalized
model approximators (such as Bayesian networks).
We describe preliminary experiments
that compare our approach with classical prioritized sweeping.
Back to Nir's
publications page
nir@cs.huji.ac.il