A Reinforcement Learning Framework for Container Selection and Ship Load Sequencing in Ports

We describe a reinforcement learning (RL) framework for selecting and sequencing containers to load onto ships in ports. The goal is to minimize an approximation of the number of crane movements require to load a given ship, known as the shuffle count. It can be viewed as a version of the assignment problem in which the sequence of assignment is of importance and the task rewards are order dependent. The proposed methodology is developed specifically to be usable on ship and yard layouts of arbitrary scale, by dividing the full problem into fixed future horizon segments and through a redefinition of the action space into a binary choice framework. Using data from real-world yard and ship layouts, we show that our approach solves the single crane version of the loading problem for entire ships with better objective values than those computed using standard metaheuristics.