Evaluation of Electricity Market Clearing Mechanisms via Reinforcement Learning: Prices, Remuneration and Competitive Dynamics
Evaluation of Electricity Market Clearing Mechanisms via Reinforcement Learning: Prices, Remuneration and Competitive Dynamics
This paper evaluates three electricity market clearing mechanisms—Pay-as-Clear (PaC), Pay-as-Bid (PaB), and the recently proposed Segmented Pay-as-Clear (SPaC)—using a Reinforcement Learning (Q-Learning) approach to model strategic bidding behavior of operators. The SPaC mechanism endogenously splits demand between two sub-markets: one for negligible marginal cost sources (renewables) and one for non-negligible marginal cost sources (fossil fuels), minimizing total system cost via a bilevel optimization. The study compares the mechanisms under two scenarios: a simplified one based on Italy's 2030 energy plan (PNIEC) with two symmetric operators, and a realistic one with ten operators derived from actual 2024 market offers.
Results show that SPaC significantly reduces inframarginal profits and price volatility compared to PaC, while maintaining efficient price signals and fair participation incentives. Under perfect competition, SPaC reduces total system cost by 57% relative to PaC. When operators learn strategic markups via Q-Learning, SPaC yields lower consumer prices than PaC and is more robust to market power than PaB. The framework provides a quantitative tool for regulators to assess market design reforms, demonstrating that SPaC can decouple renewable remuneration from fossil fuel prices without sacrificing allocative efficiency.
Highlights
- 1First application of Reinforcement Learning (Q-Learning) to evaluate the Segmented Pay-as-Clear (SPaC) market mechanism.
- 2SPaC reduces inframarginal profits and price volatility compared to Pay-as-Clear (PaC), while maintaining fair participation incentives.
- 3SPaC is more robust than Pay-as-Bid (PaB) against market power exercise in oligopolistic contexts.
- 4The framework can serve as a support tool for regulators and policymakers in market design reforms.
Methods
- MReinforcement Learning (Q-Learning) to model strategic bidding behavior of market operators.
- MBilevel optimization for the SPaC market clearing, solved as a Mathematical Program with Equilibrium Constraints (MPCC).
- MAgent-based simulation framework implemented in Julia, using BilevelJuMP package.
- MComparison of three market mechanisms (PaC, PaB, SPaC) under two scenarios: simplified PNIEC 2030 and realistic 10-operator setup.
Results
- RUnder perfect competition (marginal cost offers), SPaC reduces total system cost by 57% compared to PaC (€29,800 vs €69,000 for 1 GW demand).
- RSPaC captures about 71% of the cost reduction benefits achievable by switching from PaC to PaB, while maintaining differentiated price signals.
- RWith strategic bidding learned via Q-Learning, SPaC yields lower and less volatile consumer prices than PaC, and is more robust to market power than PaB.
- RIn the 10-operator realistic scenario, SPaC reduces the average PUN (national single price) by approximately 30% compared to PaC, while keeping producer profits adequate.
Analyze Paper
Generate insights from "Evaluation of Electricity Market Clearing Mechanisms via Rei...".