The Design and Implementation of Modern Online Programming Competitions

This paper presents a framework for the implementation of online programming competitions, including a set of principles for the design of the multiplayer game and a practical framework for the construction of the competition environment. The paper presents a successful example competition, the 2016-17 Halite challenge, and briefly mentions a second competition, the Halite II challenge, which launched in October 2017.

💡 Research Summary

The paper “The Design and Implementation of Modern Online Programming Competitions” introduces a comprehensive framework for creating successful online programming contests, derived from the authors’ hands-on experience with the Halite challenge. It identifies the growing importance of such competitions for education, recruitment, and research benchmarking, and addresses the lack of formalized design principles in this domain.

The core contribution is the Effective Programming Competition Framework (EPCoF), which is divided into two interconnected sets of principles. The first set concerns the design of the programming game itself. It mandates that the game must be simple and intuitive to learn quickly, visually appealing to attract and retain participants, computationally inexpensive to run at scale, strategically deep to prevent trivial dominant strategies, and must offer a continuous path for improvement to keep players engaged at all skill levels.

The second set of principles guides the construction of the competition infrastructure. It emphasizes a beginner-friendly platform, robust security to sandbox user-submitted code, scalable architecture to handle growing participant numbers, real-time standings to provide feedback and motivation, features that foster community collaboration (e.g., forums, shared replays), and a streamlined development environment for efficient bot iteration.

The paper uses the 2016-17 Halite competition as a detailed case study to validate EPCoF. Halite is a turn-based strategy game where 2-6 players expand territory on a toroidal grid by moving pieces and battling opponents. Despite simple rules—moving pieces in four directions and managing a “strength” stat—the game emerges as complex, involving optimization across expansion, combat, tactics, and production efficiency. The game was designed to be unsolvable, using procedurally generated maps with varying resource (production) distributions to ensure adaptability is key.

The competition infrastructure supported this by providing starter kits in multiple languages, a local testing environment, and a scalable cloud-based ranking system. Security was handled via Docker containers to isolate bot execution. A critical success factor was the real-time leaderboard using the TrueSkill algorithm and the public availability of all game replays, which drove engagement, facilitated learning, and built community. The open-source development model allowed the community to contribute, expanding language support significantly.

The analysis highlights how Halite met each EPCoF principle. For instance, its visualizer made games easy to follow, its O(n) game engine allowed millions of fast simulations, and emergent behaviors like the “non-aggression pact” demonstrated its strategic depth and meta-game evolution. The paper concludes by noting the launch of Halite II in October 2017, built on the same EPCoF principles, underscoring the framework’s utility as a reproducible blueprint for creating engaging and sustainable programming competitions.