Fine-Tuning Large Language Models for Stock Return Prediction Using Newsflow
Fine-Tuning Large Language Models for Stock Return Prediction Using Newsflow
This paper explores fine-tuning large language models (LLMs) for stock return prediction using financial newsflow, proposing a direct news-to-return framework that bypasses traditional feature extraction. The model consists of a text representation module (a pre-trained LLM) and a forecasting module (a dense layer), jointly fine-tuned to minimize mean squared error. The authors compare encoder-only (DeBERTa) and decoder-only (Mistral, Llama3) LLMs, as well as two methods for aggregating token-level embeddings: bottleneck (using an End-of-Sequence token) and aggregated (averaging all token embeddings). Experiments on real news data from North American, European, and Emerging markets show that aggregated representations generally yield better portfolio performance, particularly for decoder-only LLMs in larger universes. Mistral demonstrates the most robust performance across different markets. The LLM-based predictions significantly outperform traditional sentiment scores (FinBERT, FinVader) in backtests of long-only and long-short portfolios, highlighting the potential of fine-tuned LLMs for quantitative investing.
Highlights
- 1Proposes a novel framework for direct news-to-return prediction by fine-tuning LLMs, bypassing traditional feature extraction.
- 2Compares encoder-only (DeBERTa) and decoder-only (Mistral, Llama3) LLMs for stock return forecasting, revealing performance differences across investment universes.
- 3Introduces two simple methods (bottleneck and aggregated representations) to integrate token-level LLM embeddings into a forecasting module.
- 4Demonstrates that LLM-based return predictions outperform conventional sentiment scores (FinBERT, FinVader) in portfolio backtests.
- 5Shows that aggregated representations generally yield stronger portfolio performance, especially for decoder-only LLMs in large universes.
Methods
- MFine-tuning pre-trained LLMs (DeBERTa, Mistral-7B, Llama3-8B) with Low-Rank Adaptation (LoRA) for return prediction.
- MBottleneck representation: using an End-of-Sequence token's embedding as the sequence representation.
- MAggregated representation: averaging all token embeddings from the LLM to form the sequence representation.
- MPortfolio construction based on decile rankings of return predictions, with backtesting of long-only and long-short portfolios.
Results
- RAggregated representations generally produce higher decile returns and better portfolio performance than bottleneck representations, especially for decoder-only LLMs.
- RDecoder-only LLMs (Mistral, Llama3) lead to stronger portfolios in large universes (North America), while no consistent winner emerges in smaller universes (Europe, Emerging).
- RMistral performs most robustly across different investment universes among the three LLMs studied.
- RLLM-based return predictions outperform traditional sentiment scores (FinBERT, FinVader) in both long-only and long-short portfolio backtests.
- RHigher decile precision is more indicative of portfolio performance than lower RMSE, as predictions may have high error but still rank correctly.
Analyze Paper
Generate insights from "Fine-Tuning Large Language Models for Stock Return Predictio...".