On the Utility of External Agent Intention Predictor for Human-AI Coordination

Reaching a consensus on the team plans is vital to human-AI coordination. We suggest incorporating external models to assist humans in understanding the intentions of AI agents when the AI has no explainable plan to communicate. In this paper, we propose a twostage paradigm that first trains a Theory of Mind (ToM) model from collected offline trajectories of the target agent and utilizes the model in the process of human-AI collaboration by real-timely displaying the future action predictions of the target agent. We further implement a transformer-based predictor as the ToM model and develop an extended online human-AI collaboration platform for experiments. Experimental results validate that our ToM model can significantly improve team performance, demonstrating the potential of our paradigm in human-AI collaboration.