This paper systematically compares Transformer-based and LSTM-based models for financial time series prediction using high-frequency limit order book data from Binance Exchange. Three tasks are considered: mid-price prediction (regression of absolute prices), mid-price difference prediction (regression of price changes), and mid-price movement prediction (classification of price direction). The study evaluates multiple LSTM variants (vanilla LSTM, DeepLOB, DeepLOB-Seq2Seq, DeepLOB-Attention) and Transformer variants (vanilla Transformer, LogTrans, Reformer, Informer, Autoformer, FEDformer, Pyraformer). A new LSTM-based model called DLSTM is proposed, which integrates time series decomposition via an Inception module into the LSTM architecture for movement prediction. Additionally, Transformer architectures are adapted for movement prediction by modifying the output layer.

The experimental results reveal that Transformer-based models have a limited advantage in absolute price prediction, achieving 10-25% lower error than LSTM-based models, but the predictions are not accurate enough for practical trading. In contrast, LSTM-based models perform better and more robustly in difference prediction, with the highest out-of-sample R² reaching 11.5%. For movement prediction, the proposed DLSTM significantly outperforms all other models, achieving accuracy between 63.73% and 73.31% across different horizons. Simulated trading based on DLSTM predictions shows strong profitability and robustness, while Transformer-based models fail to generate profitable signals. The study concludes that LSTM-based models, especially DLSTM, are more suitable for financial time series prediction tasks involving price changes and movements, whereas Transformers may be better suited for absolute price forecasting but still require further improvement for practical applications.