AI in Investment Analysis: LLMs for Equity Stock Ratings

—

AI in Investment Analysis: LLMs for Equity Stock Ratings

Kassiani Papasotiriou
Srijan Sood
Shayleen Reynolds
Tucker Balch
Published on 10/30/2024
Equities
Stocks
AI
LLM
Machine learning
Fundamental analysis
Sentiment
Stock picking

This paper explores the use of Large Language Models (LLMs) for generating equity stock ratings, a task traditionally performed by financial analysts. The authors employ GPT-4-32k (v0613) with a training cutoff before the evaluation period to prevent information leakage. They design five prompting methods that incorporate varying data modalities: historical returns (Vanilla), news summaries, news sentiment scores, financial fundamentals, and a combination of fundamentals with sentiment. Ratings are evaluated by comparing them to forward return quintiles over 1, 3, 6, 12, and 18-month horizons, using Mean Absolute Error (MAE) as the metric.

The results show that all LLM-based methods outperform traditional analyst ratings in terms of average MAE, with the Fundamentals and Fundamentals + Sentiment methods achieving the best performance (MAE ~1.42 vs. analyst MAE of 1.57). Interestingly, adding news data improves short-term (1-month) predictions but harms longer-term accuracy, suggesting that news can introduce noise or bias. Sentiment scores provide a token-efficient alternative to full news summaries without significant performance loss. The study also highlights that analysts exhibit a strong bias toward buy ratings, while LLM-generated ratings are more evenly distributed.

The paper provides a reproducible framework for leveraging LLMs in investment analysis, demonstrating that even without fine-tuning, LLMs can generate more accurate and less biased stock ratings than human analysts. Future work is suggested to extend the analysis to longer horizons, incorporate additional data sources, and test newer models.

Highlights

  • 1Novel application of LLMs (GPT-4-32k) for predicting equity stock ratings, an under-explored area.
  • 2Demonstrates that incorporating financial fundamentals significantly improves rating accuracy over traditional analyst ratings.
  • 3News data enhances short-term (1-month) predictions but can introduce bias; sentiment scores offer a token-efficient alternative without performance loss.
  • 4Provides a reproducible framework for generating consistent stock ratings using LLMs without fine-tuning, evaluated by forward returns.
  • 5Addresses information leakage by using a model with a training cutoff before the evaluation period.

Methods

  • M
    Instruction-based prompting with GPT-4-32k (v0613) using chain-of-thought and few-shot examples.
  • M
    Quantile-based evaluation: ratings are compared to forward return quintiles (market-relative and sector-relative) to compute Mean Absolute Error (MAE).
  • M
    Ablation experiments with five methods: Vanilla (historical returns only), News (news summaries), Sentiment (pre-computed sentiment scores), Fundamentals (quarterly financial data), and Fundamentals + Sentiment.

Results

  • R
    The Vanilla LLM method (MAE=1.447) outperforms traditional analyst ratings (MAE=1.570) on composite forward return error.
  • R
    Fundamentals + Sentiment achieves the lowest composite MAE (1.417), followed closely by Fundamentals alone (1.421).
  • R
    News summaries improve 1-month predictions but degrade longer-term performance; sentiment scores yield similar results to summaries with fewer tokens.
  • R
    All LLM methods show higher standard deviation than analysts, indicating less consistency but better average accuracy.
  • R
    Analyst ratings are heavily biased toward buy recommendations (over 75% buy/strong buy), while LLM ratings are more balanced.
0/5

Analyze Paper

Generate insights from "AI in Investment Analysis: LLMs for Equity Stock Ratings".

Suggested Actions