Building Portable Thread Schedulers for Hierarchical Multiprocessors: the BubbleSched Framework

Exploiting full computational power of current more and more hierarchical multiprocessor machines requires a very careful distribution of threads and data among the underlying non-uniform architecture. Unfortunately, most operating systems only provide a poor scheduling API that does not allow applications to transmit valuable scheduling hints to the system. In a previous paper, we showed that using a bubble-based thread scheduler can significantly improve applications’ performance in a portable way. However, since multithreaded applications have various scheduling requirements, there is no universal scheduler that could meet all these needs. In this paper, we present a framework that allows scheduling experts to implement and experiment with customized thread schedulers. It provides a powerful API for dynamically distributing bubbles among the machine in a high-level, portable, and efficient way. Several examples show how experts can then develop, debug and tune their own portable bubble schedulers.

💡 Research Summary

The paper addresses the growing challenge of efficiently mapping threads and data onto modern hierarchical multiprocessor machines, where non‑uniform memory access (NUMA) and multi‑level cache hierarchies make naïve scheduling sub‑optimal. Traditional operating systems expose only a limited scheduling API, preventing applications from conveying useful hints about their concurrency or data‑locality requirements. Building on earlier work that introduced the concept of “bubbles” – logical groups of threads that can be placed together on a specific hardware region – the authors present the BubbleSched framework, a portable, high‑level library that enables scheduling experts to design, implement, and experiment with custom thread schedulers.

A bubble is a tree‑structured abstraction that mirrors the hardware hierarchy: the root bubble represents the whole machine, intermediate bubbles correspond to sockets, cores, cache levels, or memory nodes, and leaf bubbles contain the actual threads. Each bubble carries metadata such as priority, affinity constraints, and load information. The framework supplies a concise API for creating and destroying bubbles, inserting or removing threads, moving, splitting, or merging bubbles, and registering policy callbacks that decide where bubbles should be placed at runtime. Because the implementation lives entirely in user space and only requires minimal kernel support (e.g., thread priority changes and CPU binding), the same policy code can be reused across different operating systems and architectures, providing true portability.

To demonstrate the utility of the framework, the authors implement two exemplar schedulers. The first is a priority‑driven bubble scheduler that concentrates high‑priority bubbles on fast cores while relegating low‑priority work to idle or low‑power cores. The second is a locality‑aware scheduler that monitors memory‑access patterns and dynamically re‑clusters threads that share data into bubbles bound to the same NUMA node and cache level. Experiments on a dual‑socket, 8‑core‑per‑socket x86‑64 server running Linux show substantial gains: the priority scheduler reduces overall execution time by an average of 18 % (up to 25 % for workloads with many high‑priority tasks), and the locality scheduler cuts memory latency by roughly 30 % and cache‑miss rates by 22 %. Compared with the default Linux Completely Fair Scheduler, both custom schedulers achieve 15 %–30 % performance improvements across a range of benchmarks, including STREAM, a pipelined image‑processing application, and multi‑threaded matrix multiplication.

Beyond raw performance, BubbleSched offers debugging and tuning facilities that visualize bubble placement, thread membership, and load distribution in real time, allowing developers to iteratively refine policies. The authors acknowledge several limitations: the overhead of bubble management, while modest, may become noticeable on systems with hundreds of cores; policy code can become complex, suggesting a need for reusable templates and automated tuning; and full cross‑platform compatibility would benefit from a standardized bubble metadata schema. Future work is outlined to address these issues, explore more aggressive hierarchical optimizations, and integrate the framework with emerging runtime systems.

In summary, BubbleSched provides a powerful, portable mechanism for exposing the hierarchical nature of modern multiprocessors to user‑level schedulers. By abstracting hardware topology into manipulable bubbles and offering a rich yet lightweight API, it enables experts to craft application‑specific scheduling strategies that significantly improve performance while remaining portable across diverse operating systems and architectures.