|
|
Reinforcement Learning Ramp Metering to Balance Mainline
and Ramp Traffic Operations
ZHANG Lihui1,2, YU Hongxin1,3, XIONG Manchu1,2, HU Wenqin1, WANG Yibing1
2023, 42(4):
87-97.
DOI: 10.3969/j.issn.1674-0696.2023.04.12
Considering the traffic flow conditions of both mainline and ramp in ramp merging areas, a robust adaptive ramp metering model named Deep Reinforcement Learning-Based Adaptive Ramp Metering (DRLARM) based on deep reinforcement learning was proposed.According to traffic flow operation characteristics, a reinforcement learning reward function balancing mainline traffic efficiency and ramp queue length was constructed.To adapt to the dynamically changing traffic environment, a mixed training control model with multiple traffic flow scenarios was adopted, and simulation experiments were conducted under test scenarios such as different congestion causes, different congestion duration and different demand distribution.The average travel time A, lane occupancy ratio o, ramp queue length W and ramp loss time radio P were compared and analyzed in the case of uncontrolled, DRLARM, ALIENA, and PI-ALINEA models.The research shows that the average travel time A controlled by the DRLARM model has been saved by 22% compared to the uncontrolled working condition, slightly better than the ALIENA model, and has a similar control effect as the PI-ALINEA model does.In addition, the ramp loss time ratio P generated by the DRLARM model in different testing scenarios is relatively stable and the absolute value of ramp queue length W is shortened by about 16%, compared with the that of ALIENA model and PI-ALINEA model.The deep reinforcement learning method has taken into account both traffic efficiency and right-of-way fairness, and the trained DRLARM model exhibits good robustness under dynamic traffic conditions.
References |
Related Articles |
Metrics
|
