Autonomous Learning Agents: Layered Learning and Ad Hoc Teamwork

In order to achieve long-term autonomy in the real world, fully autonomous agents need to be able to learn, both to improve their behaviors in a complex, dynamically changing world, and to enable interaction with previously unfamiliar agents. This talk begins by presenting layered learning, a hierarchical machine learning paradigm that enables learning of complex behaviors by incrementally learning a series of sub-behaviors. Layered learning was the key deciding factor in UT Austin Villa's recent RoboCup 3D simulation league championship. The talk then introduces ad hoc teamwork as an emerging multiagent learning challenge. Ad hoc teamwork is based on the premise that as autonomous agents become capable of long-term autonomy, they will increasingly need to band together for cooperative activities with previously unfamiliar teammates. In such "ad hoc" team settings, team strategies cannot be developed a priori. Rather, an agent must learn to cooperate with new teammates on the fly. This talk reports on both theoretical and empirical ad hoc teamwork results, including from recent "pick up" RoboCup robot soccer competitions.