FedHPD: Heterogeneous Federated Reinforcement Learning via Policy Distillation

Wenzheng Jiang (National University of Defense Technology), Ji Wang (National University of Defense Technology), Xiongtao Zhang (National University of Defense Technology), Weidong Bao (National University of Defense Technology), Cheston Tan (CFAR, A*STAR), Flint Xiaofeng Fan (National University of Singapore)

Abstract

This paper focuses on Federated Reinforcement Learning (FedRL) in black-box settings with heterogeneous agents. Existing studies mostly assume agent homogeneity and knowability of internal details. To tackle these issues, we propose Federated Heterogeneous Policy Distillation (FedHPD). FedHPD uses action probability distributions as a medium for knowledge sharing among heterogeneous agents. Extensive experiments show that FedHPD achieves significant improvements across various benchmark tasks.