Diverse Heterogeneous Graph Conditioned Diffusion for Multi-Agent Teaming

Diverse multi-agent teams have the potential to solve complex tasks by learning effective teaming through reinforcement learning (RL). The high variability of interactions across team compositions poses scalability and real-world applicability challenges for online methods, highlighting the need for offline approaches that learn from pre-collected datasets. However, it is challenging to effectively leverage diverse data, adapt across team compositions using only offline data, and maintain decentralization during online deployment. To address these challenges, we present Heterogeneous Graph Conditioned Diffusion (HGCD), a multi-agent diffusion model that leverages the conditional generative modeling abilities of diffusion and heterogeneous multi-agent communication to learn generalizable policies offline, while ensuring decentralized execution online. We demonstrate the effectiveness of our method on StarCraft II Multi-Agent Challenge v2 (SMACv2) tasks, achieving superior generalization performance over prior state-of-the-art.