A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem
A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem
This paper introduces a novel deep reinforcement learning framework designed to address the financial portfolio management problem without relying on traditional financial models. The framework incorporates several innovative components: an Ensemble of Identical Independent Evaluators (EIIE) topology for decision-making, a Portfolio-Vector Memory (PVM) to maintain portfolio state information, an Online Stochastic Batch Learning (OSBL) scheme for efficient training, and an explicit reward function to optimize performance. These elements work together to provide a flexible and adaptive solution for dynamic portfolio allocation.
The methodology is implemented using three different neural network architectures—Convolutional Neural Network (CNN), basic Recurrent Neural Network (RNN), and Long Short-Term Memory (LSTM)—to instantiate the framework. These implementations are tested in back-test experiments focused on cryptocurrency markets, with trading periods set at 30 minutes. The results demonstrate that all three instances of the framework consistently outperform a range of existing portfolio-selection strategies, securing the top three positions in all experiments. Notably, the framework achieves at least a 4-fold return over 50 days, even with a high commission rate of 0.25%, highlighting its robustness and potential for high-yield automated trading in volatile markets like cryptocurrencies.
Highlights
- 1Proposes a financial-model-free Reinforcement Learning framework for portfolio management
- 2Introduces novel components: EIIE topology, Portfolio-Vector Memory, Online Stochastic Batch Learning, and explicit reward function
- 3Implements the framework with three neural network architectures (CNN, RNN, LSTM) for comparison
- 4Demonstrates superior performance in cryptocurrency market backtests, outperforming existing strategies
- 5Achieves high returns (at least 4-fold in 50 days) despite a high commission rate
Methods
- MDeep Reinforcement Learning with an Ensemble of Identical Independent Evaluators (EIIE) topology
- MPortfolio-Vector Memory (PVM) for tracking portfolio states
- MOnline Stochastic Batch Learning (OSBL) scheme for efficient training
- MBack-testing with three neural network instantiations (CNN, RNN, LSTM) in cryptocurrency markets
Results
- RAll three framework instances (CNN, RNN, LSTM) ranked top three in all back-test experiments
- ROutperformed multiple recently reviewed or published portfolio-selection strategies
- RAchieved at least 4-fold returns over 50 days in cryptocurrency trading with 30-minute periods
- RMaintained high performance despite a 0.25% commission rate in simulations
Analyze Paper
Generate insights from "A Deep Reinforcement Learning Framework for the Financial Po...".