Quantifying Agent Interaction in Multi-agent Reinforcement Learning for Cost-efficient Generalization

Yuxin Chen (University of California, Berkeley), Chen Tang (The University of Texas at Austin), Ran Tian (University of California, Berkeley), Chenran Li (University of California, Berkeley), Jinning Li (University of California, Berkeley), Masayoshi Tomizuka (University of California, Berkeley), Wei Zhan (University of California, Berkeley)

Abstract

Generalization in Multi-agent Reinforcement Learning (MARL) is challenging. Introducing a diverse set of co-play agents typically boosts the agent's generalization to unseen co-players. However, the extent to which an agent is influenced by co-players varies across scenarios and environments; thus, the improvement in generalization introduced by diversifying co-players also varies. In this work, we introduce Level of Influence (LoI), a novel metric measuring the interaction intensity among agents within a given scenario and environment. We show that LoI can effectively predict the disparities in the benefits of diversifying co-player distribution across scenarios, offering insights into optimizing training cost for varied situations. The code is available at: https://github.com/ ThomasChen98/Level-of-Influence.