Counterexample-Guided Policy Refinement in Multi-Agent Reinforcement Learning

Multi-Agent Reinforcement Learning (MARL) policies are being incorporated into a wide range of safety-critical applications. It is important for these policies to be free of counterexamples and adhere to safety requirements. We present a methodology for the counterexample-guided refinement of an optimized MARL policy with respect to given safety specifications. The proposed algorithm refines a calibrated MARL policy to become safer by eliminating counterexamples found during testing, using targeted gradient updates. We empirically validate our method on different cooperative multi-agent tasks and demonstrate that targeted gradient updates induce safety in MARL policies.