Layered Learning

Machines operating in unstructured, real-world environments is the vision of many artifical intelligence researchers. Although humans effectively filter out important information from vast sensory data and learn from their experience, researchers as yet cannot match this level of performance artificially.

The aim of this project is to develop general purpose intelligent systems that can learn and be taught to perform many different tasks autonomously by interacting with their environment. As an approach to this problem, we are interested in how machines can compute abstracted representations of their environment through direct interaction, with and without human assistance, in order to achieve some objective.

These future intelligent systems will be goal directed and adaptive, able to program themselves automatically by sensing and acting, accumulating knowledge over their lifetime. This requires a totally new computing paradigm. The challenge is to build intelligent systems that exhibit similar functionality to that of the human brain. 

What will this research achieve?

Any breakthrough in this area will have far reaching benefits given the growth and ubiquity of information and communication technology.

Who will benefit?

Applications include decision support in defence, smart transport, entertainment, image understanding, natural language understanding and autonomous robotics, to name but a few. 

What are the key features?

The layered learning project decomposes problems into modular task-hierarchies using probabilistic state-operator representations, hierarchical reinforcement learning theory and hierarchical belief propagation.

Project Status

The project has been completed and further research in this area is being continued in the Architectures for Intelligent Agent (ARIA) project.  

Research team

Bernhard Hengst (Project Leader)
Nobuyuki Morioka

Publications

2007
Safe State Abstraction and Reusable Continuing Subtasks in Hierarchical Reinforcement Learning  Bernhard Hengst, Twentieth Australian Joint Conference on Artificial Intelligence - AI07, Gold Coast, Queensland, Australia December 2007.

2005

Structural Abstraction Experiments in Reinforcement Learning (202 KB) Robert Fitch, Bernhard Hengst, Dorian Suc, Greg Calbert, Jason Scholz, The 18th Australian Joint Conference on Artificial Intelligence, Sydney Australia December 2005.

2004
Model Approximation for HEXQ Hierarchical Reinforcement Learning (312 KB) Bernhard Hengst, 15th European Conference on Machine Learning (ECML), Pisa, Italy, © Springer-Verlag.

Concurrent Discovery of Task Hierarchies (570 KB) Duncan Potts, Bernhard Hengst,  AAAI Spring Symposium on Knowledge Representation and Ontology for Autonomous Systems 2004.

Discovering Multiple Levels of a Task Hierarchy Concurrently, Duncan Potts and Bernhard Hengst
Robotics and Autonomous Systems, Volume 49, Issues 1-2, 30 November 2004, Pages 43-55 PDF (273 K)

2003

Discovering Hierarchy in Reinforcement Learning, Bernhard Hengst, PhD Thesis, Computer Science and Engineering, University of New South Wales, Sydney Australia.

Variable Resolution in Hierarchical RL ( 88 KB) Bernhard Hengst (2003) UNSW-CSE-TR #0309, National ICT Australia, School of Computer Science and Engineering, University of New South Wales.

Safe State Abstraction and Discounting in Hierarchical Reinforcement Learning (209 KB) Bernhard Hengst (2003), UNSW-CSE-TR #0308, National ICT Australia, School of Computer Science and Engineering, University of New South Wales.