FNSPID: A Comprehensive Financial News Dataset in Time Series

—

FNSPID: A Comprehensive Financial News Dataset in Time Series

Zihan Dong
Xinyu Fan
Zhiyuan Peng
Published on 2/9/2024
Equities
Stocks
United States (US)
AI
LLM
Machine learning
Sentiment
Alternative data

FNSPID (Financial News and Stock Price Integration Dataset) is a comprehensive dataset designed to bridge the gap between quantitative financial data and qualitative sentiment analysis for stock market prediction. It comprises 29.7 million stock prices and 15.7 million time-aligned financial news records for 4,775 S&P500 companies, spanning 1999 to 2023, sourced from NASDAQ, Bloomberg, Reuters, Benzinga, and Lenta. The dataset includes raw news text, four types of summarizations (LexRank, Luhn, LSA, TextRank), and sentiment scores for a subset of 50 stocks labeled by ChatGPT on a 1-5 scale. Missing sentiment values are handled via exponential decay. The authors validate FNSPID through extensive experiments with six deep learning models (LSTM, CNN, GRU, RNN, Transformer, TimesNet) for stock price prediction. Key findings show that larger training datasets significantly boost accuracy (6.29% average R² improvement from 5 to 25 stocks), and high-quality sentiment (ChatGPT) provides a modest 0.2% improvement for transformer models, while lower-quality sentiment (TextBlob) degrades performance. The Transformer model achieves the highest accuracy (R²=0.988). FNSPID is publicly available with reproducible code, offering a valuable resource for financial sentiment analysis, LLM fine-tuning, and time-series forecasting research.

Highlights

  • 1Introduces FNSPID, a large-scale dataset combining 29.7 million stock prices and 15.7 million financial news records for 4,775 S&P500 companies from 1999 to 2023.
  • 2Dataset uniquely integrates time-aligned news and stock prices with sentiment scores from ChatGPT, enabling advanced financial analysis.
  • 3Demonstrates that larger training datasets significantly improve stock price prediction accuracy across multiple deep learning models.
  • 4Shows that high-quality sentiment scores (ChatGPT) modestly enhance transformer-based model performance, while lower-quality sentiment (TextBlob) can degrade accuracy.
  • 5Provides a reproducible data collection and update procedure, facilitating ongoing research.

Methods

  • M
    Data collection via web scraping (NASDAQ) and integration of existing datasets (Bloomberg, Reuters, Benzinga, Lenta) for news; Yahoo Finance API for stock prices.
  • M
    News summarization using LexRank, Luhn, LSA, and TextRank algorithms to handle token limits.
  • M
    Sentiment quantification using ChatGPT with a 1-5 scale, and exponential decay to fill missing sentiment values.
  • M
    Deep learning models: LSTM, CNN, GRU, RNN, Transformer, and TimesNet for stock price prediction with and without sentiment features.

Results

  • R
    Transformer model achieved highest accuracy (R²=0.988) with 50 stocks, outperforming LSTM (R²=0.856) and GRU (R²=0.827).
  • R
    Average R² improvement of 6.29% when increasing training from 5 to 25 stocks across all models.
  • R
    ChatGPT sentiment scores provided a 0.2% improvement in Transformer R², while TextBlob sentiment caused a 1.16% decrease.
  • R
    Dataset includes 402,546 news items with sentiment scores for 50 top S&P500 stocks.
  • R
    FNSPID covers 4,775 stocks with 15.7 million news articles and 29.7 million price records over 24 years.
0/5

Analyze Paper

Generate insights from "FNSPID: A Comprehensive Financial News Dataset in Time Serie...".

Suggested Actions