Trading-R1: Financial Trading with LLM Reasoning via Reinforcement Learning
Trading-R1: Financial Trading with LLM Reasoning via Reinforcement Learning
Trading-R1 is a financial reasoning large language model designed for trading decisions, trained on a novel 100k-sample dataset (Tauric-TR1-DB) that integrates technical data, fundamentals, news, sentiment, and macroeconomic indicators over 18 months across 14 major equities. The model employs a three-stage easy-to-hard curriculum: first, supervised fine-tuning (SFT) with reverse reasoning distillation from proprietary LLMs teaches structured thesis writing; second, reinforcement learning (RL) with evidence-grounding rewards reduces hallucinations; third, RL with volatility-aware trading labels aligns decisions with market outcomes. The labeling scheme discretizes forward returns into five actions (strong sell to strong buy) using multi-horizon Sharpe-like signals normalized by volatility, with asymmetric quantile thresholds to reflect market drift. Policy optimization uses Group Relative Policy Optimization (GRPO) to stabilize training without a value model. Evaluated on held-out data for NVDA, AAPL, MSFT, GOOGL, AMZN, and SPY, Trading-R1 outperforms both open-source (Qwen, LLaMA) and proprietary (GPT-4.1, DeepSeek) models in cumulative return, Sharpe ratio, and maximum drawdown, while generating interpretable, evidence-backed investment theses. The work addresses key challenges in financial AI: grounding reasoning in diverse data, ensuring structural discipline, and producing actionable, risk-aware decisions.
Highlights
- 1Introduces Trading-R1, a financial reasoning LLM trained via a three-stage easy-to-hard curriculum combining supervised fine-tuning and reinforcement learning.
- 2Constructs Tauric-TR1-DB, a large-scale 100k-sample financial reasoning corpus spanning 18 months, 14 equities, and five heterogeneous data sources.
- 3Proposes reverse reasoning distillation to extract hidden reasoning traces from proprietary LLMs for supervised training.
- 4Develops a volatility-driven discretization method for generating trading labels based on multi-horizon Sharpe-like signals.
- 5Demonstrates improved risk-adjusted returns and lower drawdowns compared to both open-source and proprietary models on six major equities and ETFs.
Methods
- MReverse reasoning distillation: reconstructs reasoning paths from black-box LLMs by inferring steps from final outputs and elaborating per-modality contributions.
- MThree-stage easy-to-hard curriculum: interleaves supervised fine-tuning (SFT) and reinforcement learning fine-tuning (RFT) to progressively teach structure, evidence grounding, and decision making.
- MVolatility-driven label generation: computes composite signals from 3/7/15-day forward returns normalized by rolling volatility, then assigns five-class labels via asymmetric percentile thresholds.
- MGroup Relative Policy Optimization (GRPO): optimizes policy without a value model by computing advantages from group-relative rewards across sampled trajectories.
Results
- RTrading-R1 achieves higher cumulative returns and Sharpe ratios compared to baselines including Qwen-4B, GPT-4.1, LLaMA-3.3, and DeepSeek on NVDA, AAPL, and MSFT.
- RThe model exhibits lower maximum drawdowns than most baselines, indicating improved risk management.
- RAblation studies show that the full three-stage training (SFT + RFT) outperforms variants with only SFT or only RL.
- RThe volatility-aware labeling scheme produces a realistic asymmetric target distribution (15% strong buy, 32% buy, 38% hold, 12% sell, 3% strong sell) that reflects market upward drift.
- RTrading-R1 generates structured, evidence-based investment theses that support interpretable and disciplined trading decisions.
Analyze Paper
Generate insights from "Trading-R1: Financial Trading with LLM Reasoning via Reinfor...".