This paper introduces multi-horizon forecasting models for limit order book (LOB) data using deep learning, specifically sequence-to-sequence (Seq2Seq) and attention mechanisms. Unlike standard single-step prediction, the proposed encoder-decoder architectures generate a forecasting path over multiple horizons. The encoder is based on DeepLOB, a state-of-the-art network combining CNNs, Inception modules, and LSTM, while the decoder uses either Seq2Seq or attention to produce multi-step outputs. Experiments on the FI-2010 benchmark and a large LSE dataset (169M samples) show that the models achieve competitive performance at short horizons and significantly outperform at long horizons (e.g., k=100) due to the autoregressive structure that feeds short-term predictions into future estimates. Attention weights reveal that recent LOB observations are most important, explaining why Seq2Seq does not suffer from long input sequences. The paper also addresses the computational bottleneck of recurrent encoder-decoder models by leveraging Graphcore's Intelligent Processing Units (IPUs). Benchmarks show IPUs reduce training time to about 15% of GPU time, offering a practical solution for large-scale LOB data. Transfer learning experiments further demonstrate the models' robustness, as they generalize well to unseen stocks, suggesting universal features in high-frequency LOB data.