Evaluating the Role of Interactivity on Improving Transparency in Autonomous Agents

Autonomous agents are increasingly being deployed amongst human end-users. Yet, human users often have little knowledge of how these agents work or what they will do next. This lack of transparency has already resulted in unintended consequences during AI use: a concerning trend which is projected to increase with the proliferation of autonomous agents. To curb this trend and ensure safe use of AI, assisting users in establishing an accurate understanding of agents that they work with is essential. In this work, we present AI Teacher, a user-centered Explainable AI framework to address this need for autonomous agents that follow a Markovian policy. Our framework first computes salient instructions of agent behavior by estimating a user's mental model and utilizing algorithms for sequential decision-making. Next, in contrast to existing solutions, these instructions are presented interactively to the endusers, thereby enabling a personalized approach to improving AI transparency. We evaluate our framework, with emphasis on its interactive features, through experiments with human participants. The experiment results suggest that, relative to non-interactive approaches, interactive teaching can both reduce the amount of time it takes for humans to create accurate mental models of these agents and is subjectively preferred by human users.