MATLight: Traffic Signal Coordinated Control Algorithm based on Heterogeneous-Agent Mirror Learning with Transformer

In order to better handle the issue of real-time multi-intersection traffic signal coordinated control, we expect that multi-agent decisionmaking can benefit from the advantages of large sequence models. In this paper, we propose a method for multi-intersection traffic signal coordinated control based on heterogeneous-agent mirror learning and Transformer to sequential multi-agent cooperative decision. First, multi-intersection traffic signal control is modeled as a sequential problem based on the heterogeneous-agent mirror learning framework. We convert real-time multi-intersection traffic signal control into a multi-agent sequential decision-making process. It completely capitalizes on the surprising connection between the multi-agent reinforcement learning decision process and sequential model prediction. And it provides strong theoretical guarantees. Then the Transformer sequence model is used to cleverly implement the sequential update scheme to learn the optimal traffic signal coordination control strategy online with a new training paradigm. The proposed method has theoretical policy promotion and convergence, alleviates the credit assignment problem in the process of multi-intersection traffic signal coordinated control, reduces the complexity of the joint policy optimization, and improves the learning efficiency of few-shot samples. We used LibSignal, a unified framework for traffic signal control tasks, for comparison testing. According to experimental results, our method can significantly improve the efficiency and performance of few-shot online learning, outperform the baseline methods in both network-level and arterial coordination, and simplify the complexity of algorithm implementation.