Neuro-Symbolic World Models for Adapting to Open World Novelty

Jonathan C. Balloch (Georgia Institute of Technology), Zhiyu Lin (Georgia Institute of Technology), Xiangyu Peng (Georgia Institute of Technology), Mustafa Hussain (Georgia Institute of Technology), Aarun Srinivas (Georgia Institute of Technology), Robert Wright (Georgia Tech Research Institute), Julia M. Kim (Georgia Institute of Technology), Mark O. Riedl (Georgia Institute of Technology)

Abstract

Most reinforcement learning (RL) methods assume that the world is a closed, fixed process, when in reality most real world problems are open, changing over time. To address this, we introduce World-Cloner, an end-to-end trainable neuro-symbolic world model that learns an efficient symbolic model of transitions and uses this world model to improve novelty adaptation. We show that the symbolic world model helps WorldCloner adapt its policy more efficiently than neural-only reinforcement learning methods.