关键词: 交通工程； 匝道合流；多智能体强化学习；规则约束；奖励机制；PPO 算法

Abstract: The ramp merging problem in mixed traffic was transformed into a rule-based multi-agent reinforcement learning problem. By designing a framework that integrated rule constraints and learning capabilities, autonomous vehicles on ramps and main roads adapting to human driving behaviors in a coordinated manner were realized, thereby maximizing traffic efficiency. The main research contents were as follows: firstly, a multi-dimensional reward mechanism for a single agent was proposed, which built a comprehensive reward system including speed rewards, lane entry penalties, headway distance penalties and collision penalties. Through dynamic speed mapping, lane occupancy cost calculation, and dynamic safety distance assessment, a safety and efficiency constraint system was formed. Secondly, a regional reward mechanism for multi-agent coordination was designed, which constructed a regional reward model based on vehicle interaction relationships. By utilizing a dynamically topology-adaptive parameter sharing strategy and adjusting reward weights based on communication connection status, the collaborative efficiency of multi-agent systems was enhanced. Thirdly, the multi-agent adaptability of the PPO algorithm was improved by optimizing the core process of reinforcement learning and introducing a clipping objective function of interaction factors to improve training stability and convergence efficiency. Experiments show that the proposed framework outperforms multiple kinds of advanced methods in terms of merging success rate, collision rate and other indicators, providing a safe and efficient solution for ramp merging in mixed traffic scenarios.

Key words: traffic engineering; ramp merging; multi-agent reinforcement learning; rule constraint; reward mechanism; PPO algorithm