Integrating Large Language Models and Reinforcement Learning for Sentiment-Driven Quantitative Trading
Integrating Large Language Models and Reinforcement Learning for Sentiment-Driven Quantitative Trading
This paper develops a sentiment-driven quantitative trading system that integrates FinGPT-generated sentiment signals with traditional technical indicators using both a conventional rule-based approach and a reinforcement learning (RL) framework based on the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm. The study uses a universe of 44 S&P 500 stocks from 2018 to 2025, with out-of-sample testing from 2024 to 2025. News articles are summarized using LLaMA 3.1 8B and then processed by FinGPT for sentiment classification, while technical indicators include RSI, VWAP, MACD, and volatility measures. The rule-based strategy combines signals via linear weighting and constructs a long-short quintile portfolio, while the RL agent learns to dynamically allocate weights in a long-only portfolio with transaction costs.
The results show that sentiment signals provide added value: in the rule-based strategy, positive sentiment weights reduce maximum drawdowns, though they slightly lower returns and Sharpe ratios. The RL-driven strategy outperforms the buy-and-hold benchmark, achieving a 23.65% annualized return, Sharpe ratio of 1.38, and Sortino ratio of 1.96 out-of-sample, despite a 52.3% annual turnover and 10bps transaction costs. Factor decomposition reveals that the RL strategy's returns are largely explained by market beta (0.91) with defensive, large-cap, and anti-momentum tilts, and the alpha is insignificant after costs. The study also addresses look-ahead bias by using a FinGPT model with a November 2023 cutoff and masking company names and dates in pre-2024 news. Overall, the paper demonstrates that RL provides a promising framework for integrating heterogeneous signals in dynamic trading environments, though transaction costs remain a critical consideration.
Highlights
- 1Integrates FinGPT sentiment signals with technical indicators using both rule-based and RL (TD3) frameworks.
- 2Demonstrates that sentiment signals from FinGPT add value, reducing drawdowns in rule-based strategies and improving risk-adjusted returns in RL strategies.
- 3RL-driven TD3 strategy outperforms buy-and-hold with a Sharpe ratio of 1.38 and 23.65% annualized return out-of-sample.
- 4Mitigates look-ahead bias by using pre-training cutoff (Nov 2023) and masking company names/dates in pre-2024 news.
- 5Provides a comprehensive comparison between conventional linear weighting and RL-based dynamic signal integration.
Methods
- MSentiment analysis using FinGPT (fine-tuned LLM) on summarized financial news, producing daily sentiment scores with confidence.
- MTechnical indicator computation: RSI, VWAP, MACD, Garman-Klass volatility, and volume pressure.
- MRule-based strategy: linear combination (equal weight) of z-score normalized technical and sentiment signals, ranking stocks into quintiles for long-short portfolio.
- MReinforcement learning: Twin Delayed Deep Deterministic Policy Gradient (TD3) with twin critics, target smoothing, and delayed policy updates; state includes 7 features plus previous weights; action is softmax-projected portfolio weights.
Results
- RRule-based long-short with sentiment weight 0.5 yields OOS annual return 16.66%, Sharpe 1.37; higher sentiment weight reduces drawdown but also return.
- RFama-French 5-factor + momentum model explains only 2-3% of rule-based strategy returns, suggesting alpha beyond conventional factors.
- RRL-driven TD3 strategy achieves OOS annual return 23.65%, Sharpe 1.38, Sortino 1.96, with 52.3% turnover and 10bps transaction costs.
- RRL strategy's factor decomposition shows R²=0.65, significant loadings on market (0.91), size (-0.17), and momentum (-0.14), with insignificant alpha after costs.
- RTransaction costs significantly erode rule-based returns (e.g., from 13.84% to 3.66% for sentiment weight 0), highlighting the importance of cost-aware design.
Analyze Paper
Generate insights from "Integrating Large Language Models and Reinforcement Learning...".