Multimodal Forecasting for Commodity Prices Using Spectrogram-Based and Time Series Representations
Multimodal Forecasting for Commodity Prices Using Spectrogram-Based and Time Series Representations
The paper presents Spectrogram-Enhanced Multimodal Fusion (SEMF), a novel framework for multivariate time series forecasting that combines spectral and temporal representations. SEMF transforms the target commodity price series into Morlet wavelet spectrograms, which are encoded by a Vision Transformer to capture localized frequency-aware features. Exogenous variables (e.g., Treasury yields, exchange rates, market indices) are processed by a Transformer encoder with reversible instance normalization. A bidirectional cross-attention module integrates these modalities, enabling joint modeling of time-frequency patterns and multivariate dependencies. The model is trained to predict multiple horizons (1, 3, 7, 14, 21, 35 days) simultaneously using a shared MLP head.
Evaluated on coal, gold, and steel price forecasting, SEMF consistently outperforms seven baselines including LSTM, iTransformer, TimesNet, PatchTST, VisionTS, and ViT-num-spec across RMSE, RMAE, MAPE, and R² metrics. Horizon-wise analysis reveals that SEMF maintains stable accuracy across all forecast lengths, whereas baselines exhibit error growth at longer horizons. Ablation studies confirm the importance of each component: Morlet spectrograms outperform line plots and STFT; Transformer encoder surpasses MLP for exogenous variables; bidirectional cross-attention improves over single-direction fusion. The framework's horizon-consistent behavior supports reliable decision-making in procurement, inventory management, and risk mitigation under volatile market conditions.
Highlights
- 1Proposes Spectrogram-Enhanced Multimodal Fusion (SEMF), a dual-path framework integrating spectral and temporal representations for commodity price forecasting.
- 2Employs Morlet wavelet spectrograms to capture non-stationary, multi-scale dynamics in financial time series.
- 3Introduces a bidirectional cross-attention fusion module to align spectral patterns with exogenous contextual signals.
- 4Achieves consistent improvements over seven baselines across multiple horizons and commodities, demonstrating robustness and practical relevance.
Methods
- MMorlet wavelet transform to generate spectrograms from target time series.
- MVision Transformer (ViT) encoder for spectrogram feature extraction.
- MTransformer-based encoder with reversible instance normalization (RevIN) for exogenous variables.
- MBidirectional cross-attention mechanism for multimodal fusion.
Results
- RSEMF outperforms LSTM, iTransformer, TimesNet, PatchTST, VisionTS, and ViT-num-spec on coal, gold, and steel price forecasting.
- RHorizon-wise analysis shows SEMF maintains stable performance across 1- to 35-day forecasts, unlike baselines with increasing errors.
- RAblation studies confirm that Morlet spectrograms, Transformer exogenous encoder, and bidirectional cross-attention each contribute to performance gains.
- ROptimal configuration uses patch size 8 and wavelet scale 128, with input sequence length 120.
Analyze Paper
Generate insights from "Multimodal Forecasting for Commodity Prices Using Spectrogra...".