Alpha-R1: Alpha Screening with LLM Reasoning via Reinforcement Learning
Alpha-R1: Alpha Screening with LLM Reasoning via Reinforcement Learning
Alpha-R1 introduces a reinforcement learning-based reasoning model for dynamic alpha screening in quantitative finance. The framework addresses the challenge of factor decay and regime shifts by using an 8B-parameter LLM trained via Group Relative Policy Optimization (GRPO) to evaluate factor relevance based on real-time market conditions. The model processes heterogeneous data—including technical indicators and financial news—into structured semantic descriptions, enabling context-aware factor selection. A fixed linear model provides stable return predictions, while the LLM acts as a gating network to activate or deactivate factors, reducing model misspecification and estimation noise in non-stationary markets.
Empirical results on the CSI 300 and CSI 1000 asset pools demonstrate that Alpha-R1 consistently outperforms traditional machine learning methods (e.g., XGBoost, PCA) and generic reasoning LLMs (e.g., DeepSeek-R1, Claude 3.7 Sonnet) in terms of cumulative returns, Sharpe ratio, and maximum drawdown. The model also exhibits robust zero-shot generalization to out-of-domain small-cap stocks. Ablation studies confirm the importance of both news and price data, as well as the superiority of semantic gating over heuristic alternatives. The work bridges the gap between static quantitative models and dynamic market environments, offering a transparent and adaptive approach to factor investing.
Highlights
- 1Proposes Alpha-R1, an 8B-parameter reasoning model trained via reinforcement learning for context-aware alpha screening, which dynamically activates or deactivates factors based on semantic alignment with market conditions.
- 2Introduces a novel framework that bridges static quantitative models and dynamic market environments by synthesizing heterogeneous information (macroeconomic indicators, news narratives) for regime-aware factor selection.
- 3Adapts the RLHF paradigm to finance by replacing subjective human preferences with objective market-based reward signals, using Group Relative Policy Optimization (GRPO) for stable training.
- 4Demonstrates consistent outperformance over traditional ML baselines and generic LLMs across multiple asset pools, with improved robustness to alpha decay and interpretable factor selection.
Methods
- MReinforcement Learning with Group Relative Policy Optimization (GRPO): Uses a critic-free optimization to fine-tune the LLM reasoning core, with a multi-component reward function combining market performance (excess returns) and structural penalties (parsimony, validity).
- MSemantic Profiling and State Description: Maps quantitative factor backtest results and market data into structured textual descriptions (factor profiles, market state) using LLMs, enabling context-aware reasoning.
- MContext-Conditioned Sparse Linear Model: Interprets the framework as a gating mechanism where the LLM selects relevant factors based on semantic alignment, while a fixed linear model provides stable return predictions.
- MSlot Rotation and VWAP Execution: Implements a practical portfolio construction with multiple sub-portfolios and volume-weighted average price execution to mitigate turnover costs and ensure market realism.
Results
- ROn the CSI 300 (in-domain), Alpha-R1 achieves a Sharpe ratio of 1.62 and cumulative return of 12.99%, outperforming all baselines including XGBoost (-10.03%), PPO (0.89%), and DeepSeek-R1 (-5.98%).
- ROn the CSI 1000 (out-of-domain), Alpha-R1 achieves a Sharpe ratio of 4.03 and cumulative return of 42.49%, demonstrating strong zero-shot generalization compared to A2C (11.80%) and PPO (-6.44%).
- RAlpha-R1 exhibits lower maximum drawdown (6.76% on CSI 300, 9.25% on CSI 1000) compared to generic LLMs (e.g., Claude 3.7 Sonnet: 13.58% on CSI 300), indicating better risk management.
- RAblation studies confirm that both news and price components contribute to performance, and the semantic gating mechanism outperforms heuristic gating strategies.
Analyze Paper
Generate insights from "Alpha-R1: Alpha Screening with LLM Reasoning via Reinforceme...".