Peter Stone's Selected Publications

Classified by TopicClassified by Publication TypeSorted by DateSorted by First Author Last NameClassified by Funding Source


Learning Predictive State Representations

Learning Predictive State Representations.
Satinder Singh, Michael L. Littman, Nicholas K. Jong, David Pardoe, and Peter Stone.
In Proceedings of the Twentieth International Conference on Machine Learning, August 2003.
ICML-2003

Download

[PDF]316.3kB  [postscript]319.2kB  

Abstract

We introduce the first algorithm for learning predictive state representations PSRs, which are a way of representing the state of a controlled dynamical system. The state representation in a PSR is a vector of predictions of tests, where tests are sequences of actions and observations said to be true if and only if all the observations occur given that all the actions are taken. The problem of finding a good PSR---one that is a sufficient statistic for the dynamical system---can be divided into two parts: 1) discovery of a good set of tests, and 2) learning to make accurate predictions for those tests. In this paper, we present detailed empirical results using a gradient-based algorithm for addressing the second problem. Our results demonstrate several sample systems in which the algorithm learns to make correct predictions and several situations in which the algorithm is less successful. Our analysis reveals challenges that will need to be addressed in future PSR learning algorithms.

BibTeX Entry

@InProceedings{ICML03,
        author="Satinder Singh and Michael L.~Littman and Nicholas K.~Jong and David Pardoe and Peter Stone",
        title="Learning Predictive State Representations",
        booktitle="Proceedings of the Twentieth International Conference on Machine Learning",
        year="2003",month="August",
        abstract={
                  We introduce the first algorithm for learning
                  predictive state representations PSRs, which are a
                  way of representing the state of a controlled
                  dynamical system.  The state representation in a PSR
                  is a vector of predictions of tests, where tests are
                  sequences of actions and observations said to be
                  true if and only if all the observations occur given
                  that all the actions are taken.  The problem of
                  finding a good PSR---one that is a sufficient
                  statistic for the dynamical system---can be divided
                  into two parts: 1) discovery of a good set of tests,
                  and 2) learning to make accurate predictions for
                  those tests.  In this paper, we present detailed
                  empirical results using a gradient-based algorithm
                  for addressing the second problem.  Our results
                  demonstrate several sample systems in which the
                  algorithm learns to make correct predictions and
                  several situations in which the algorithm is less
                  successful.  Our analysis reveals challenges that
                  will need to be addressed in future PSR learning
                  algorithms.
        },
        wwwnote={<a href="http://www.hpl.hp.com/conferences/icml03/">ICML-2003</a>},
}

Generated by bib2html.pl (written by Patrick Riley ) on Mon Mar 25, 2024 00:05:17