Autonomous Vehicle Cut-In Algorithm for Lane-Merging Scenarios via Policy-Based Reinforcement Learning Nested Within Finite-State Machine
- Authors
- Hwang, Seulbin; Lee, Kibeom; Jeon, Hyeongseok; Kum, Dongsuk
- Issue Date
- Oct-2022
- Publisher
- IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
- Keywords
- Safety; Reinforcement learning; Autonomous vehicles; Vehicles; Decision making; Automata; Stochastic processes; Autonomous vehicle; lane-merge; cut-in; deep reinforcement learning; finite-state machine
- Citation
- IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS, v.23, no.10, pp.17594 - 17606
- Journal Title
- IEEE TRANSACTIONS ON INTELLIGENT TRANSPORTATION SYSTEMS
- Volume
- 23
- Number
- 10
- Start Page
- 17594
- End Page
- 17606
- URI
- https://scholarworks.bwise.kr/gachon/handle/2020.sw.gachon/85976
- DOI
- 10.1109/TITS.2022.3153848
- ISSN
- 1524-9050
- Abstract
- Lane-merging scenarios pose highly challenging problems for autonomous vehicles due to conflicts of interest between the human-driven and cutting-in autonomous vehicles. Such conflicts become severe when traffic increases, and cut-in algorithms suffer from a steep trade-off between safety and cut-in performance. In this study, a reinforcement learning (RL)-based cut-in policy network nested within a finite state machine (FSM)--which is a high-level decision maker, is proposed to achieve high cut-in performance without sacrificing safety. This FSM-RL hybrid approach is proposed to obtain 1) a strategic and adjustable algorithm, 2) optimal safety and cut-in performance, and 3) robust and consistent performance. In the high-level decision making algorithm, the FSM provides a framework for four cut-in phases (ready for safe gap selection, gap approach, negotiation, and lane-change execution) and handles the transitions between these phases by calculating the collision risks associated with target vehicles. For the lane-change phase, a policy-based deep-RL approach with a soft actor-critic network is employed to get optimal cut-in performance. The results of simulations show that the proposed FSM-RL cut-in algorithm consistently achieves a high cut-in success rate without sacrificing safety. In particular, as the traffic increases, the cut-in success rate and safety are significantly improved over existing optimized rule-based cut-in algorithms and end-to-end RL algorithm.
- Files in This Item
- There are no files associated with this item.
- Appears in
Collections - ETC > 1. Journal Articles
![qrcode](https://api.qrserver.com/v1/create-qr-code/?size=55x55&data=https://scholarworks.bwise.kr/gachon/handle/2020.sw.gachon/85976)
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.