Deep Reinforcement Learning for Active High Frequency Trading
Deep Reinforcement Learning for Active High Frequency Trading
This paper presents a pioneering framework for active high-frequency trading using Deep Reinforcement Learning (DRL), specifically the Proximal Policy Optimization algorithm. The research focuses on trading Intel Corporation stock by training DRL agents on three months of high-frequency Limit Order Book data, with the last month reserved for validation and a subsequent month for testing. To enhance training efficiency, the authors maximize the signal-to-noise ratio by selecting only samples with the largest price changes. Hyperparameters are optimized using Sequential Model Based Optimization, and three distinct state characterizations based on LOB meta-features are explored to assess agent performance.
The study demonstrates that DRL agents can develop dynamic representations of the market environment, identifying and exploiting occasional regularities in the data to formulate long-term profitable trading strategies. Despite the highly stochastic and non-stationary nature of high-frequency trading, the agents achieve stable positive returns, highlighting the framework's effectiveness. This work contributes to quantitative finance by showcasing the potential of end-to-end DRL in creating adaptive and robust trading systems, with implications for improving algorithmic trading strategies in volatile markets.
Highlights
- 1First end-to-end Deep Reinforcement Learning framework for active high-frequency trading
- 2Uses Proximal Policy Optimization algorithm to train agents on Intel Corporation stock
- 3Maximizes signal-to-noise ratio by selecting training samples with largest price changes
- 4Agents create dynamic representations and exploit data regularities for profitable strategies
Methods
- MDeep Reinforcement Learning (DRL) with Proximal Policy Optimization algorithm
- MTraining on high-frequency Limit Order Book data with three contiguous months
- MSequential Model Based Optimization for hyperparameter tuning
- MThree different state characterizations based on LOB meta-features
Results
- RAgents learn trading strategies that produce stable positive returns in a stochastic environment
- RDynamic representation of the underlying environment enables identification of data regularities
- RStrategies are profitable long-term despite non-stationary market conditions
- RValidation and testing on separate months of data confirm robustness
Analyze Paper
Generate insights from "Deep Reinforcement Learning for Active High Frequency Tradin...".