Abstract: In order to meet the requirements of ship stowage plans, reduce the number of crane movements, and enhance terminal operational efficiency, the block relocation problem (BRP) considering stowage plan was studied, which was based on the traditional BRP and also considered the impact of stowage plan on block relocation. To minimize the number of container movements for this problem, a DQN algorithm was designed for solving it. Meanwhile, in order to improve the solving performance of the algorithm, the threshold values based on heuristic algorithms and a new reward function were designed on the basis of original algorithm to improve the algorithm. By comparing with experimental results in other literature, the results show that in terms of computational results, the improved DQN algorithm outperforms the optimal results of current heuristic algorithms in all cases and the larger the scale, the better the results. Moreover, in terms of training time, the improved DQN algorithm significantly outperforms the unimproved one, and the larger the scale, the more significant the time saved.

Key words: traffic and transportation engineering; ocean shipping; block relocation problem (BPR); stowage plan; deep Q-network (DQN) algorithm

摘要： 为了满足船舶配载图的要求，减少场桥翻箱次数，提高码头运行效率，对考虑船舶配载图的集装箱翻箱问题进行了研究。此问题是在传统集装箱翻箱问题的基础上，又考虑到船舶配载图对翻箱的影响。为了求解此问题的最小翻箱次数，设计了DQN算法进行求解，同时为了提高算法求解的性能，又在原算法的基础上设计了基于启发式算法的阈值和全新的奖励函数以改进算法。通过与其它文献中的实验结果进行对比，结果显示：在计算结果上，改进的DQN算法在各个算例上的结果均优于目前各个启发式算法的最优结果，并且规模越大，结果越好；在训练时间上，改进的DQN算法极大的优于未改进的DQN算法，并且规模越大，节省的时间也更显著。

关键词: 交通运输工程；海运；集装箱翻箱；船舶配载图；DQN算法