Peter Stone's Selected Publications

Classified by TopicClassified by Publication TypeSorted by DateSorted by First Author Last NameClassified by Funding Source


Hierarchical Model-Based Reinforcement Learning: Rmax + MAXQ

Hierarchical Model-Based Reinforcement Learning: Rmax + MAXQ.
Nicholas K. Jong and Peter Stone.
In Proceedings of the Twenty-Fifth International Conference on Machine Learning, July 2008.
ICML 2008

Download

[PDF]157.5kB  [postscript]370.2kB  

Abstract

Hierarchical decomposition promises to help scale reinforcement learning algorithms naturally to real-world problems by exploiting their underlying structure. Model-based algorithms, which provided the first finite-time convergence guarantees for reinforcement learning, may also play an important role in coping with the relative scarcity of data in large environments. In this paper, we introduce an algorithm that fully integrates modern hierarchical and model-learning methods in the standard reinforcement learning setting. Our algorithm, \textscR-maxq, inherits the efficient model-based exploration of the \textscR-max algorithm and the opportunities for abstraction provided by the MAXQ framework. We analyze the sample complexity of our algorithm, and our experiments in a standard simulation environment illustrate the advantages of combining hierarchies and models.

BibTeX Entry

@InProceedings{ICML08-jong,
    author="Nicholas K.\ Jong and Peter Stone",
    title="Hierarchical Model-Based Reinforcement Learning: {Rmax} + {MAXQ}",
    booktitle="Proceedings of the Twenty-Fifth International Conference on Machine Learning",
    month="July",year="2008",
    abstract={ Hierarchical decomposition promises to help scale
               reinforcement learning algorithms naturally to
               real-world problems by exploiting their underlying
               structure.  Model-based algorithms, which provided the
               first finite-time convergence guarantees for
               reinforcement learning, may also play an important
               role in coping with the relative scarcity of data in
               large environments.  In this paper, we introduce an
               algorithm that fully integrates modern hierarchical
               and model-learning methods in the standard
               reinforcement learning setting.  Our algorithm,
               \textsc{R-maxq}, inherits the efficient model-based
               exploration of the \textsc{R-max} algorithm and the
               opportunities for abstraction provided by the MAXQ
               framework.  We analyze the sample complexity of our
               algorithm, and our experiments in a standard
               simulation environment illustrate the advantages of
               combining hierarchies and models.  },
    wwwnote={<a href="http://icml2008.cs.helsinki.fi/">ICML 2008</a>},
}

Generated by bib2html.pl (written by Patrick Riley ) on Mon Mar 25, 2024 00:05:16