Learning Robust Helpful Behaviors in Two-Player Cooperative Atari Environments

We study the problem of learning helpful behavior, specifically, learning to cooperate with differently-skilled and diverse partners in the context of two-player, cooperative Atari games. We show robust performance of these so-called Helper-AIs when paired with different kinds of partners (both human and artificial agents), including partners that they have not previously encountered during training. In particular, while pairing an expert AI with a non-expert AI leads to performance that is worse than when pairing the nonexpert AI with a copy of itself, these Helper-AIs provide a substantial boost in joint performance.