Fast Adaption by Policy Deviation Integral Meta-reinforcement Learning with Applications to High-speed Trains Operation

Current deep reinforcement learning (DRL)-based to optimize trajectories for real-world high-speed rail (HSR) face two issues: 1) a driver-centric Markov decision process (MDP) with sparse rewards and 2) single-trajectory optimization (i.e., single-task), poorly suited for real-world HSR scenarios that demand rapid adaptation to changing conditions (i.e., multi-task). To address these issues, we propose two innovations. First, a trajectory loop optimization (RTLO)-centric MDP that directly computes rewards from trajectory states, providing dense rewards. Second, a policy deviation integral meta-reinforcement learning (PDIMRL) method that enhances multi-task learning by leveraging HSR inter-task similarities, while the initial policy of the new task is linearly adjusted by the policy deviation integral between tasks' sub-optimal policies. Experiments demonstrate that 1) compared to existing driver-centric MDP, RTLO is 16.8× faster for single task training, and 2) based on RTLO-centric MDP, PDIMRL requires 2.3× fewer meta-training iterations than benchmark meta-RL methods.