This paper presents a reinforcement learning approach to market making, a trading problem where an agent provides liquidity by quoting bid and ask prices. The authors develop a high-fidelity limit order book simulator using historical data from 10 equities across 5 venues, capturing millisecond-level updates and 5 levels of order book depth. They address past criticisms of temporal-difference (TD) learning in this domain by showing that eligibility traces (e.g., Q(λ), SARSA(λ)) effectively handle reward attribution and partial observability, unlike one-step methods that struggled in prior work. The agent uses a linear combination of tile codings (LCTC) to represent state, combining independent tile codings for agent-state (inventory, quoting distances) and market-state (spread, mid-price move, order imbalance, etc.), which accelerates learning by exploiting variable independence. A key contribution is the design of an asymmetrically dampened reward function that penalizes speculative profits from inventory appreciation while retaining losses, thereby encouraging spread capture and reducing inventory risk. The authors systematically evaluate learning algorithms (Q-learning, SARSA, R-learning, and variants), reward functions, and state representations. Their consolidated agent, using SARSA(λ) with asymmetrically dampened reward and LCTC, outperforms fixed benchmark strategies, a basic RL agent, and the online learning approach of Abernethy and Kale (2013) in out-of-sample tests across multiple stocks. The results demonstrate that TD learning with careful reward and state design can produce viable market making strategies in realistic simulated markets.