Carbon Trading Supply Chain Management Based on Constrained Deep Reinforcement Learning

Reducing carbon emissions remains a formidable challenge in supply chain management. However, conventional numerical simulations based on heuristics often struggle to address the complex coupling of ordering decisions and carbon emissions-complicated further by high-dimensional observation, multiple constraints, and carbon quota requirements. Constrained DRL leverages the highdimensional representation capabilities and robust decision-making under constraints, making it particularly suitable for supply chain management involving carbon trading. To address these challenges, this paper proposes a simulation framework grounded in Constrained Markov Decision Processes (CMDP), incorporating constrained deep reinforcement learning (DRL). Specifically, we develop a Double Order algorithm based on PPO-Lagrangian (DOPPOL) to simultaneously optimize business and carbon costs. Experimental results demonstrate that DOPPOL outperforms traditional (𝑠, 𝑆) methods under fluctuating demand, effectively balancing cost optimization and emission reductions. Furthermore, integrating carbon trading into supply chain operations allows companies to adapt both ordering decisions and emissions, thereby enhancing operational efficiency. Then we highlight the pivotal role of carbon pricing in business contracts: rational pricing not only helps regulate carbon emissions but also reduces overall costs. Our findings contribute to the broader endeavor of mitigating climate change and promoting sustainable supply chain practices.