An adaptive dual-level reinforcement learning approach for optimal trade execution
- Authors
- Kim, Soohan; Kim, Jimyeong; Sul, Hong Kee; Hong, Youngjoon
- Issue Date
- Oct-2024
- Publisher
- Elsevier Ltd
- Keywords
- Markov decision process; Optimal trade execution; Proximal policy optimization; Reinforcement learning; Volume-weighted average price
- Citation
- Expert Systems with Applications, v.252
- Journal Title
- Expert Systems with Applications
- Volume
- 252
- URI
- https://scholarworks.bwise.kr/cau/handle/2019.sw.cau/74197
- DOI
- 10.1016/j.eswa.2024.124263
- ISSN
- 0957-4174
1873-6793
- Abstract
- The purpose of this research is to devise a tactic that can closely track the daily cumulative volume-weighted average price (VWAP) using reinforcement learning while minimizing the deviation from the VWAP. Previous studies often choose a relatively short trading horizon to implement their models, making it difficult to accurately track the daily cumulative VWAP since the stock price movement is often insignificant within the short trading horizon. On the other hand, training reinforcement learning models directly over a longer, daily horizon is burdensome due to extensive sequence length. Hence, there is a need for a method that can divide the long daily horizon into smaller, more manageable segments. We propose a method that leverages the U-shaped pattern of intraday stock trade volumes and uses Proximal Policy Optimization (PPO) as the learning algorithm. Our method follows a dual-level approach: a Transformer model that captures the overall (global) distribution of daily volumes in a U-shape, and a LSTM model that handles the distribution of orders within smaller (local) time intervals. The results from our experiments suggest that this dual-level architecture improves cumulative VWAP tracking accuracy compared to previous reinforcement learning approaches. The key finding is that explicitly accounting for the U-shaped intraday volume pattern leads to better performance in approximating the cumulative daily VWAP. This has implications for developing trading strategies that need to efficiently track VWAP over a full trading day. © 2024 Elsevier Ltd
- Files in This Item
- There are no files associated with this item.
- Appears in
Collections - College of Business & Economics > School of Business Administration > 1. Journal Articles
![qrcode](https://api.qrserver.com/v1/create-qr-code/?size=55x55&data=https://scholarworks.bwise.kr/cau/handle/2019.sw.cau/74197)
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.