MM-DREX (Multimodal-Driven Dynamic Routing of LLM Experts for Financial Trading) is a novel framework that addresses the non-stationarity and complexity of financial markets by decoupling market state perception from strategy execution. It uses a vision-language model (VLM) as a dynamic router to process candlestick charts, time-series data, and textual information, allocating real-time weights to four specialized trading experts (trend, reversal, breakout, positioning). These experts generate fine-grained sub-strategies, and their outputs are aggregated into a diversified portfolio. The system is trained via a hybrid paradigm: supervised fine-tuning (SFT) first teaches the router to classify market regimes (uptrend, downtrend, consolidation), followed by reinforcement learning (RL) that optimizes expert policies for risk-adjusted returns. This two-stage approach mitigates gradient interference and enables efficient exploration.

Extensive experiments on a multimodal dataset covering 62 assets (US/A-share stocks, ETFs, futures, cryptocurrencies) from 2017 to 2025 show that MM-DREX significantly outperforms 15 baselines, including traditional technical indicators, deep learning models (LSTM, Transformer), deep RL methods (PPO, SAC, DQN), and state-of-the-art financial LLMs (FinAgent, FinMem). Key metrics—total return, Sharpe ratio, and maximum drawdown—are consistently superior across all asset classes. Ablation studies confirm the critical roles of dynamic routing (vs. static or uniform weighting) and visual modality (candlestick patterns). Risk analysis demonstrates robust crisis resilience: during six black-swan events (e.g., COVID-19 crash, 2022 rate hikes, 2025 trade war), MM-DREX limits losses and often generates positive returns while the S&P 500 suffers double-digit declines. The framework also includes an interpretability module that traces routing decisions and expert contributions, enhancing transparency for regulatory compliance and strategy refinement.