FinTSB is a comprehensive and practical benchmark for financial time series forecasting (FinTSF) designed to address three systemic limitations in existing evaluations: diversity gap, standardization deficit, and real-world mismatch. The benchmark collects and pre-processes 15 years of historical stock data from multiple markets, categorizing stock movement patterns into four types (uptrend, downtrend, volatility, extreme events) to ensure coverage of diverse market conditions. It provides 20 datasets, each containing 300 stocks over 250 trading days, with careful tokenization and normalization to prevent data leakage. FinTSB standardizes evaluation across three metric dimensions—ranking (e.g., IC, RankIC), portfolio (e.g., annualized return, Sharpe ratio, max drawdown), and error (e.g., MSE, MAE)—and incorporates real-world trading constraints such as transaction fees, short-selling restrictions, and trading halts to ensure practical relevance.

The benchmark evaluates over 30 methods spanning six backbone categories: classic strategies, machine learning, deep learning, reinforcement learning, generative models, and large language models. Extensive experiments reveal key insights: deep learning methods generally outperform traditional approaches, but no single method dominates across all patterns; for instance, momentum strategies excel in uptrends but fail in downtrends and volatile periods. Generative and LLM-based methods show promise in extreme events due to their ability to model uncertainty and leverage textual data. Transaction costs significantly impact portfolio returns, especially for high-turnover strategies, underscoring the importance of realistic simulation. FinTSB provides a unified, user-friendly, and lightweight pipeline to facilitate fair comparisons and advance the field of financial time series forecasting.