Diverse Approaches to Optimal Execution Schedule Generation

We present the first application of MAP-Elites, a quality-diversity algorithm, to trade execution. Rather than searching for a single optimal policy, MAP-Elites generates a diverse portfolio of regime-specialist strategies indexed by liquidity and volatility conditions. Individual specialists achieve 8-10% performance improvements within their behavioural niches, while other cells show degradation, suggesting opportunities for ensemble approaches that combine improved specialists with the baseline PPO policy. Results indicate that quality-diversity methods offer promise for regime-adaptive execution, though substantial computational resources per behavioural cell may be required for robust specialist development across all market conditions. To ensure experimental integrity, we develop a calibrated Gymnasium environment focused on order scheduling rather than tactical placement decisions. The simulator features a transient impact model with exponential decay and square-root volume scaling, fit to 400+ U.S. equities with $R^2>0.02$ out-of-sample. Within this environment, two Proximal Policy Optimization architectures - both MLP and CNN feature extractors - demonstrate substantial improvements over industry baselines, with the CNN variant achieving 2.13 bps arrival slippage versus 5.23 bps for VWAP on 4,900 out-of-sample orders ($21B notional). These results validate both the simulation realism and provide strong single-policy baselines for quality-diversity methods.

💡 Research Summary

The paper tackles the optimal execution (OE) problem by combining reinforcement learning (RL) with a quality‑diversity (QD) approach, specifically MAP‑Elites, and evaluates the resulting strategies against strong industry baselines. The authors first construct a high‑fidelity Gymnasium‑based back‑testing environment that simulates order scheduling at a one‑minute resolution for over 400 US equities. Central to the simulator is a calibrated transient market‑impact model derived from the propagator framework: impact decays exponentially (kernel G(ℓ)=G₀e^{‑ℓ/τ}) and scales with a square‑root participation‑rate law (f(q,V)=γ(q/V)^β, β≈0.5). Cross‑validation yields out‑of‑sample R² > 0.02, confirming that the model captures realistic price dynamics without over‑fitting.

Within this environment, the agent observes a rich state vector (remaining inventory, elapsed time, price, spread, volatility, market volume, order‑flow imbalance, etc.) and at each minute selects a trade size (or participation rate). The reward combines implementation shortfall and VWAP‑relative slippage, directly incentivising cost‑efficient execution. Two Proximal Policy Optimization (PPO) agents are trained: one with a multilayer perceptron (MLP) and another with a convolutional neural network (CNN) that extracts temporal patterns from the state matrix. On a held‑out test set of 4,900 orders (≈ $21 B notional), the CNN‑PPO achieves an average arrival slippage of 2.13 bps, a 59 % reduction compared with the VWAP benchmark’s 5.23 bps. This result validates the realism of the calibrated impact model and establishes a robust single‑policy baseline.

The novel contribution lies in applying MAP‑Elites, a QD algorithm that maintains a diverse archive of high‑performing policies indexed by behavioural descriptors. The authors define a two‑dimensional descriptor space: market liquidity and volatility. MAP‑Elites evolves separate elites for each cell, encouraging specialization to distinct market regimes. Empirical findings show that most specialist elites outperform the baseline PPO by 8‑10 % within their niche, while some cells experience degradation, highlighting challenges in regime classification and data sparsity. The approach demonstrates the potential of ensemble execution—combining regime‑specific specialists with a generalist PPO—to capture the best of both worlds. However, the per‑cell evolutionary process is computationally intensive, raising concerns about scalability.

The paper’s contributions are fourfold: (1) First application of QD methods to financial execution, generating a portfolio of regime‑specific policies. (2) Validation of RL under an empirically calibrated transient impact model, with CNN‑PPO delivering a 2.13 bps slippage versus 5.23 bps for VWAP. (3) Release of an open‑source calibrated Gymnasium environment to foster reproducible research. (4) Insight into the trade‑off between diversity and computational cost, suggesting future work on meta‑learning, richer descriptor sets, and online adaptation.

Limitations include the reliance on a two‑dimensional descriptor space, the high computational budget required for MAP‑Elites, and the focus on minute‑level dynamics (the model may not transfer to sub‑second execution). Future directions proposed are (a) improving sample efficiency via transfer or meta‑learning across cells, (b) expanding descriptors to capture market directionality and order‑flow bursts, and (c) integrating live market data for online updating of the elite archive. Overall, the study demonstrates that realistic impact‑aware RL can surpass traditional benchmarks and that quality‑diversity algorithms hold promise for building adaptable, regime‑aware execution strategies.