Financial Trading as a Game: A Deep Reinforcement Learning Approach

—

Financial Trading as a Game: A Deep Reinforcement Learning Approach

Chien Yi Huang
Published on 7/8/2018
FX
Deep learning
Reinforcement learning
High frequency trading
Forex system

This paper presents a novel approach to financial trading by framing it as a game and applying deep reinforcement learning techniques. The authors develop a Markov Decision Process model specifically designed for trading tasks and implement it using a modified Deep Recurrent Q-Network algorithm. Key innovations include an action augmentation technique that provides feedback signals for all possible actions, allowing the agent to use greedy policies instead of traditional epsilon-greedy exploration methods, which proves particularly effective in financial market contexts.

The methodology incorporates several practical modifications to standard deep reinforcement learning approaches, including a substantially smaller replay memory (hundreds rather than millions of entries) and longer sequence sampling for recurrent neural network training. These adaptations not only improve performance in financial trading scenarios but also reduce computational requirements, enabling more efficient online learning. The complete algorithm is validated on spot foreign exchange markets, demonstrating the practical viability of this approach for developing automated trading systems that can potentially generate consistent profits from financial markets.

Highlights

  • 1Proposes a Markov Decision Process (MDP) model tailored for financial trading tasks
  • 2Introduces modifications to deep reinforcement learning algorithms for financial market applications
  • 3Develops an action augmentation technique to reduce reliance on random exploration
  • 4Implements a small replay memory and longer sequence sampling for efficiency
  • 5Validates the approach on spot foreign exchange markets with empirical performance

Methods

  • M
    Markov Decision Process (MDP) modeling for financial trading
  • M
    Deep Recurrent Q-Network (DRQN) algorithm with custom modifications
  • M
    Action augmentation technique to provide feedback for all actions
  • M
    Small replay memory implementation and longer sequence sampling for recurrent neural networks

Results

  • R
    The modified DRQN algorithm demonstrates strong empirical performance in financial trading
  • R
    Action augmentation enables use of greedy policies instead of epsilon-greedy exploration
  • R
    Small replay memory and longer sequence sampling reduce computational requirements
  • R
    The approach shows practical applicability in spot foreign exchange markets
0/5

Analyze Paper

Generate insights from "Financial Trading as a Game: A Deep Reinforcement Learning A...".

Suggested Actions