TradeTrap: Are LLM-based Trading Agents Truly Reliable and Faithful?
TradeTrap: Are LLM-based Trading Agents Truly Reliable and Faithful?
TradeTrap is a unified evaluation framework designed to systematically stress-test LLM-based autonomous trading agents under adversarial conditions. The paper decomposes trading agents into four core components—market intelligence, strategy formulation, portfolio and ledger handling, and trade execution—and injects controlled perturbations (data fabrication, MCP tool hijacking, prompt injection, memory poisoning, state tampering) into each component. Experiments are conducted in a closed-loop historical backtesting environment using real U.S. equity market data (NASDAQ-100) with identical initial conditions, enabling fair and reproducible comparisons across two agent architectures: adaptive (tool-calling) and procedural (pipeline-driven).
The results reveal that small, localized perturbations can propagate through the decision pipeline, leading to extreme concentration, runaway exposure, and large portfolio drawdowns. Adaptive agents are highly opportunistic but vulnerable to information-channel attacks (e.g., fake news and MCP hijacking), which degrade risk-adjusted performance and induce strategic paralysis. Procedural agents exhibit greater resistance to noisy inputs but are catastrophically affected by state or memory corruption, such as memory poisoning causing persistent underperformance and state tampering converting them into single-asset strategies or accumulating massive short positions. The study highlights that current LLM-based trading agents can be systematically misled at the system level, and the choice between agent architectures involves a trade-off between upside capture and the attack surface one is willing to harden. TradeTrap provides a reproducible benchmark with nine quantitative metrics and open-source code to facilitate future robustness evaluations.
Highlights
- 1Proposes TradeTrap, a unified framework to stress-test LLM-based trading agents under adversarial perturbations across four core components.
- 2Identifies that small perturbations in a single component can propagate through the decision loop, causing extreme concentration, runaway exposure, and large drawdowns.
- 3Reveals that adaptive (tool-calling) agents are more vulnerable to information-channel attacks, while procedural (pipeline) agents are more robust to noisy inputs but highly susceptible to state/memory corruption.
- 4Demonstrates that state tampering can convert adaptive agents into single-asset buy-and-hold strategies and cause procedural agents to accumulate catastrophic short positions.
- 5Provides a reproducible evaluation protocol with nine quantitative metrics and open-source code for benchmarking agent robustness.
Methods
- MDecomposition of trading agents into four functional components: market intelligence, strategy formulation, portfolio/ledger handling, and trade execution.
- MControlled perturbation injection (data fabrication, MCP tool hijacking, prompt injection, memory poisoning, state tampering) with single-variable isolation.
- MClosed-loop historical backtesting on real U.S. equity market data (NASDAQ-100) with identical initial conditions ($5,000 capital, zero positions).
- MEvaluation using nine quantitative metrics: Total Return, Annualized Return, Maximum Drawdown, Volatility, Position Utilization, Sharpe Ratio, Calmar Ratio, Average and Maximum Position Concentration.
Results
- RData fabrication (fake news) reduces adaptive agent total return from 11.59% to 5.26% and Sharpe from 4.34 to 3.20, while procedural agents show milder degradation (return 5.83%, Sharpe 6.76).
- RMCP tool hijacking induces strategic paralysis: adaptive agent hallucinates phantom positions and ceases trading after a fabricated crash-rebound cycle.
- RPrompt injection drops adaptive agent total return from 7.81% to 0.89% and increases maximum concentration to 99.98%; procedural agent return falls from 0.91% to 0.57%.
- RMemory poisoning causes persistent underperformance: adaptive agent total return drops from 7.81% to 1.88%, procedural from 0.99% to -0.17%.
- RState tampering converts adaptive agent into a single-asset buy-and-hold (return 9.83% with high concentration) and causes procedural agent to accumulate short positions, resulting in -61.02% total return and 91.97% max drawdown.
Analyze Paper
Generate insights from "TradeTrap: Are LLM-based Trading Agents Truly Reliable and F...".