FGLight: Learning Neighbor-level Information for Traffic Signal Control

In recent years, multi-agent reinforcement learning (MARL) methods have increasingly been applied to traffic signal control and have achieved some success. However, most of existing MARL methods often underemphasize the heterogeneity in neighborhoodlevel information of the same agent. This results in highly sensitive performances and a long learning process. To address this challenge, we propose FGLight, a novel Feudal MARL method for traffic signal control. FGLight leverages Adaptive Graph Attention Networks (AGAT) to dynamically model the interactive relationships between intersections. Through adaptive neighbor selection and weight-based attention mechanisms, AGAT dynamically assigns importance weights to neighbor-level information, thereby improving the accuracy of local policies by more effectively exploiting neighborhood information. Moreover, FGLight introduces a Smooth Hysteretic Deep Q-Network (SHDQN) based on an optimistic assumption mechanism, which enhances the stability of the global policy. We conducted experiments on both synthetic and real-world datasets, and the results demonstrate that, compared to several state-of-the-art MARL methods, FGLight performs better as the complexity of the road network increases, exhibiting faster convergence and greater policy stability.