TLOB: A Novel Transformer Model with Dual Attention for Price Trend Prediction with Limit Order Book Data
TLOB: A Novel Transformer Model with Dual Attention for Price Trend Prediction with Limit Order Book Data
This paper introduces TLOB, a novel transformer model with dual attention mechanisms for price trend prediction using limit order book (LOB) data. TLOB employs separate self-attention over the temporal axis (LOB snapshots) and the spatial axis (LOB features), followed by an MLPLOB block, to capture both time-wise dependencies and cross-feature interactions. The authors also propose MLPLOB, a simple isotropic MLP architecture that alternates feature-mixing and temporal-mixing layers, demonstrating that even a straightforward design can surpass state-of-the-art (SoTA) performance. Additionally, a new labeling method is introduced that decouples the smoothing window from the prediction horizon, removing bias present in prior approaches.
Extensive experiments on the FI-2010 benchmark, NASDAQ Tesla/Intel stocks, and a Bitcoin dataset show that TLOB achieves average F1-score improvements of 3.7, 1.3/7.7, and 1.1 over SoTA, respectively. MLPLOB also performs competitively, especially on shorter horizons. The study further investigates the declining predictability of stock prices over time, finding a 6.68 F1-score drop, consistent with increasing market efficiency. When trend thresholds are set to the average spread (reflecting transaction costs), performance deteriorates, underscoring the difficulty of translating predictions into profitable trading. An ablation study confirms the necessity of both temporal and spatial attention in TLOB. The work provides insights into the evolving landscape of financial AI and sets a strong foundation for future research.
Highlights
- 1Proposes TLOB, a transformer with dual attention (temporal and spatial) for LOB-based price trend prediction.
- 2Introduces MLPLOB, a simple MLP-based model that surpasses state-of-the-art, challenging the need for complex architectures.
- 3Develops a new labeling method that decouples smoothing window from prediction horizon, removing horizon bias.
- 4Achieves state-of-the-art F1-score improvements on FI-2010 (+3.7 avg), Tesla/Intel (+1.3/+7.7), and Bitcoin (+1.1) datasets.
- 5Empirically shows declining stock price predictability over time and evaluates trend thresholds based on average spread.
Methods
- MDual-attention transformer (TLOB) with separate self-attention over time steps and LOB features, plus MLPLOB feed-forward blocks.
- MMLPLOB: isotropic MLP architecture with alternating feature-mixing and temporal-mixing MLP layers.
- MBilinear normalization layer to handle non-stationarity and magnitude disparities in LOB data.
- MNew labeling method: decouples smoothing window k from horizon h, using average of past and future mid-prices over independent windows.
Results
- RTLOB outperforms state-of-the-art on FI-2010 by average 3.7 F1-score across horizons 10,20,50,100.
- ROn Tesla and Intel, TLOB improves F1 by 1.3 and 7.7 respectively; on Bitcoin, by 1.1 average F1.
- RMLPLOB matches or exceeds complex models, showing simple architectures can be competitive.
- RStock price predictability declined by 6.68 F1-score over time, consistent with efficient market hypothesis.
- RUsing average spread as trend threshold reduces performance, highlighting challenge of translating predictions into profitable strategies.
Analyze Paper
Generate insights from "TLOB: A Novel Transformer Model with Dual Attention for Pric...".