Learning Flexible Heterogeneous Coordination With Capability-Aware Shared Hypernetworks

Cooperative heterogeneous multi-agent tasks require agents to effectively coordinate their behaviors while accounting for their relative capabilities. Learning-based solutions span between two extremes with opposing tradeoffs: i) shared-parameter solutions, which encode diverse behaviors within a single architecture by assigning an ID to each agent, are sample-efficient but result in limited behavioral diversity; ii) independent solutions, which learn a separate policy for each agent, show greater behavioral diversity but lack sample-efficiency. Prior work has also explored selective parameter-sharing, allowing for a compromise between diversity and efficiency. None of these approaches, however, effectively generalize to unseen agents or teams. We present Capability-Aware Shared Hypernetworks (CASH), a novel architecture for heterogeneous multi-agent coordination that generates behavioral diversity while staying sample-efficient via soft parameter-sharing hypernetworks. CASH allows the team to learn common strategies using a shared encoder, which are then adapted according to the team's capabilities with a hypernetwork, allowing for zero-shot generalization to unseen teams and agents. We conduct experiments across two heterogeneous coordination tasks and three learning paradigms (imitation learning, on-and off-policy reinforcement learning). Results show that CASH outperforms baseline architectures in success rate and sample efficiency when evaluated on unseen teams and agents despite using 60-80% fewer learnable parameters.