Symplex: Learning Social Norm Hierarchies by Combining Autonomous Exploration and Expert Imitation

In this paper, we introduce SYMPLEX (Symbolic Policy Learning from Experts/Exploration), an interactive framework that learns complex hierarchies of behavioral norms as interpretable logical constraints through a combination of autonomous exploration and expert imitation. The approach ensures that learned constraints are interpretable for human oversight, generalizable for transfer to similar environments, and defeasible-enabling adaptation to novel behaviors and facilitating the learning of exceptions in dynamic domains. We demonstrate the utility of our approach in a traffic simulation environment using a neuro-symbolic implementation of SYMPLEX that interleaves a Deep Q-Learning (DQL) component for policy optimization through goal-directed domain exploration, with interactive Inductive Logic Programming (ILP) for examplebased symbolic constraint generation. At each iteration, inferred constraints are imposed on the DQL via penalty terms appended to the reward function, allowing the system to form exceptions to previously-learned constraints. We illustrate SYMPLEX's ability to identify concise human-readable constraints in complex environments, and evidence the efficacy of learning norms as defeasible constraints. Additionally, we exemplify the benefits of using an interactive rule induction system in expediting convergence to accurate norms.