Peter Stone's Selected Publications

Classified by TopicClassified by Publication TypeSorted by DateSorted by First Author Last NameClassified by Funding Source


Transfer Learning via Inter-Task Mappings for Temporal Difference Learning

Transfer Learning via Inter-Task Mappings for Temporal Difference Learning.
Matthew E. Taylor, Peter Stone, and Yaxin Liu.
Journal of Machine Learning Research, 8(1):2125–2167, 2007.
Available from journal's web page.

Download

[PDF]499.9kB  [postscript]831.3kB  

Abstract

Temporal difference (TD) learning has become a popular reinforcement learning technique in recent years. TD methods, relying on function approximators to generalize learning to novel situations, have had some experimental successes and have been shown to exhibit some desirable properties in theory, but the most basic algorithms have often been found slow in practice. This empirical result has motivated the development of many methods that speed up reinforcement learning by modifying a task for the learner or helping the learner better generalize to novel situations. This article focuses on generalizing across tasks, thereby speeding up learning, via a novel form of transfer using handcoded task relationships. We compare learning on a complex task with three function approximators, a cerebellar model arithmetic computer (CMAC), an artificial neural network (ANN), and a radial basis function (RBF), and empirically demonstrate that directly transferring the action-value function can lead to a dramatic speedup in learning with all three. Using transfer via inter-task mapping (tvitm), agents are able to learn one task and then markedly reduce the time it takes to learn a more complex task. Our algorithms are fully implemented and tested in the RoboCup soccer Keepaway domain.

BibTeX Entry

@Article{JMLR07-taylor,
  author           = "Matthew E.\ Taylor and Peter Stone and Yaxin Liu",
  title            = "Transfer Learning via Inter-Task Mappings for Temporal Difference Learning",
  journal          = "Journal of Machine Learning Research",
  year             = "2007",
  volume           = "8",
  number           = "1",
  pages            = "2125--2167",
  abstract         = "Temporal difference (TD) learning has become a popular reinforcement learning technique in recent years. TD methods, relying on function approximators to generalize learning to novel situations, have had some experimental successes and have been shown to exhibit some desirable properties in theory, but the most basic algorithms have often been found slow in practice. This empirical result has motivated the development of many methods that speed up reinforcement learning by modifying a task for the learner or helping the learner better generalize to novel situations. This article focuses on generalizing across tasks, thereby speeding up learning, via a novel form of transfer using handcoded task relationships. We compare learning on a complex task with three function approximators, a cerebellar model arithmetic computer (CMAC), an artificial neural network (ANN), and a radial basis function (RBF), and empirically demonstrate that directly transferring the action-value function can lead to a dramatic speedup in learning with all three. Using transfer via inter-task mapping (tvitm), agents are able to learn one task and then markedly reduce the time it takes to learn a more complex task. Our algorithms are fully implemented and tested in the RoboCup soccer Keepaway domain.",
  wwwnote = {Available from <a href="http://jmlr.csail.mit.edu/papers/v8/taylor07a.html">journal's web page</a>.},
}

Generated by bib2html.pl (written by Patrick Riley ) on Mon Mar 25, 2024 00:05:10