Predicting Day-Ahead Stock Returns using Search Engine Query Volumes: An Application of Gradient Boosted Decision Trees to the S&P 100
Predicting Day-Ahead Stock Returns using Search Engine Query Volumes: An Application of Gradient Boosted Decision Trees to the S&P 100
This paper investigates whether Google search query volumes can predict day-ahead stock returns for S&P 100 constituents from 2005 to 2017. Using gradient boosted decision trees, the author models the binary outcome of a stock's return exceeding the index median. Features include cumulative historical returns over 13 horizons, percentage changes in search volume index (SVI) over the past 5 days, and generic information (industry, weekday, trading volume change). Models are trained on rolling 2-year windows and tested on the subsequent year, with 11 non-overlapping test periods. Seven model variants are evaluated, combining different feature sets.
The results show that all feature sets contain predictive information, with average AUC values between 54.2% and 56.7%, significantly above random chance. A simple statistical arbitrage strategy—long the 5 stocks with highest predicted probability of outperformance and short the 5 with lowest—yields annual returns exceeding 57% before transaction costs. The best performing model, using all features, achieves a cumulative return of over 1500% over 11 years, far surpassing the S&P 100 index. The study provides evidence that internet search behavior contains valuable signals for short-term stock return prediction, challenging weak and semi-strong forms of market efficiency. However, transaction costs are not considered, and the practical profitability may be lower.
Highlights
- 1Demonstrates that Google search query volumes contain predictive information for day-ahead stock returns of S&P 100 constituents.
- 2Achieves average AUC between 54.2% and 56.7% using gradient boosted decision trees, significantly better than random.
- 3A simple trading strategy based on model predictions yields annual returns exceeding 57% before transaction costs.
- 4Combining search volume data with historical returns and generic features (industry, weekday, volume) improves performance.
- 5Challenges weak and semi-strong form market efficiency by showing predictive power of publicly available data.
Methods
- MGradient boosted decision trees (GBM) for binary classification of stock outperformance vs. index median.
- MFeature engineering: cumulative historical returns (1-240 days), SVI percentage changes (1-5 days), and generic features (industry, weekday, trading volume change).
- MRolling window validation: 2-year training, 1-year test sets from 2005-2017, non-overlapping test periods.
- MStatistical arbitrage strategy: long top 5 predicted outperformers, short bottom 5 predicted underperformers daily.
Results
- RBest model (all features) achieves cumulative return >1500% over 11 years, vs. S&P 100 return of ~100%.
- RModels using only search volume data (SVI) yield positive but lower returns; combining data sources improves performance.
- RAverage AUC across all models ranges from 54.2% to 56.7%, indicating consistent predictive ability.
- RAnnualized returns of the best ensemble exceed 57% before transaction costs.
- RAll three data sources (returns, SVI, generic) contribute non-zero predictive value.
Analyze Paper
Generate insights from "Predicting Day-Ahead Stock Returns using Search Engine Query...".