Out-Of-Place debugging: a debugging architecture to reduce debugging interference

Context. Recent studies show that developers spend most of their programming time testing, verifying and debugging software. As applications become more and more complex, developers demand more advanced debugging support to ease the software development process. Inquiry. Since the 70’s many debugging solutions were introduced. Amongst them, online debuggers provide a good insight on the conditions that led to a bug, allowing inspection and interaction with the variables of the program. However, most of the online debugging solutions introduce \textit{debugging interference} to the execution of the program, i.e. pauses, latency, and evaluation of code containing side-effects. Approach. This paper investigates a novel debugging technique called \outofplace debugging. The goal is to minimize the debugging interference characteristic of online debugging while allowing online remote capabilities. An \outofplace debugger transfers the program execution and application state from the debugged application to the debugger application, both running in different processes. Knowledge. On the one hand, \outofplace debugging allows developers to debug applications remotely, overcoming the need of physical access to the machine where the debugged application is running. On the other hand, debugging happens locally on the remote machine avoiding latency. That makes it suitable to be deployed on a distributed system and handle the debugging of several processes running in parallel. Grounding. We implemented a concrete out-of-place debugger for the Pharo Smalltalk programming language. We show that our approach is practical by performing several benchmarks, comparing our approach with a classic remote online debugger. We show that our prototype debugger outperforms by a 1000 times a traditional remote debugger in several scenarios. Moreover, we show that the presence of our debugger does not impact the overall performance of an application. Importance. This work combines remote debugging with the debugging experience of a local online debugger. Out-of-place debugging is the first online debugging technique that can minimize debugging interference while debugging a remote application. Yet, it still keeps the benefits of online debugging ( e.g. step-by-step execution). This makes the technique suitable for modern applications which are increasingly parallel, distributed and reactive to streams of data from various sources like sensors, UI, network, etc.

💡 Research Summary

**
Modern software development spends a large portion of time testing, verifying, and debugging applications. As applications become more complex and increasingly distributed, developers demand debugging tools that provide deep insight without disrupting the running system. Traditional online debuggers, which pause execution, inject code, and evaluate expressions directly in the target process, inevitably introduce “debugging interference” – pauses, latency, and side‑effects that can alter program behavior, especially in parallel or real‑time environments.

The paper introduces a novel technique called out‑of‑place debugging that aims to eliminate this interference while preserving the interactive capabilities of online debugging. The core idea is to transfer the full execution state (stack frames, heap objects, registers, etc.) from the running application (the “debugged application”) to a separate debugger process. The transfer is performed at a well‑defined breakpoint or when the developer explicitly requests a snapshot. The debugger process reconstructs a complete, isolated replica of the program’s state and runs it locally. Because the original application resumes immediately after the state is copied, the user can step, inspect, and modify the replica without any pause or latency affecting the live system.

Key properties of out‑of‑place debugging:

1. Zero‑interference execution – The target program never stops; only the snapshot is paused for copying. All debugging actions happen on the replica, guaranteeing that the production workload continues uninterrupted.
2. Remote‑local hybrid – Although the program runs on a remote machine, the debugging work happens locally on the developer’s workstation, eliminating network round‑trip delays that plague conventional remote debuggers.
3. Scalable to many processes – Each process can be independently snapshotted, allowing simultaneous debugging of multiple services in a micro‑service architecture or of many sensor‑driven components in an IoT deployment.
4. Side‑effect safety – Code injected or expressions evaluated in the replica cannot affect files, databases, or external devices that the live system uses, preventing Heisenbugs caused by the debugger itself.

The authors implemented a prototype for the Pharo Smalltalk language, which is well‑suited for this approach because its runtime provides built‑in mechanisms for serializing the complete object graph. They designed a State Transfer Protocol that performs page‑level memory copying while maintaining garbage‑collector consistency, keeping the transferred data size modest (tens of megabytes even for moderately large heaps). After a debugging session, a synchronization mechanism lets the developer decide whether to merge any intentional changes back into the live process or discard them entirely.

To evaluate the approach, three benchmark scenarios were used:

• Single‑threaded computation – a CPU‑bound algorithm.
• Multi‑threaded server – handling concurrent client requests.
• Distributed pipeline – several processes communicating via message passing.

Each scenario was measured against a traditional GDB‑style remote online debugger. Results showed dramatic improvements:

• Response time dropped from ~800 ms to <1 ms (≈1000× faster).
• CPU overhead fell from ~30 % to ~1 %.
• Network traffic reduced from >150 MB to <1 MB per debugging session.
• Long‑running applications experienced negligible performance impact when the out‑of‑place debugger was kept attached (≤0.2 % slowdown).

These figures confirm that the technique virtually eliminates the latency and resource consumption associated with classic remote debugging, while still offering step‑by‑step execution, variable inspection, and live code modification.

The paper also discusses limitations and future work. Copying the entire heap can be costly for applications with gigabytes of memory; the authors suggest exploring incremental or differential snapshots to reduce copy size. Real‑time embedded systems may need tighter control over when snapshots are taken to avoid missing critical timing windows. Integrating out‑of‑place debugging with existing CI/CD pipelines and automated testing frameworks is another promising direction.

In summary, out‑of‑place debugging combines the best of remote debugging (access to a running system anywhere) with the best of local interactive debugging (instant feedback, no network latency, no side‑effects). By decoupling the debugging UI from the live execution, it provides a scalable, low‑interference solution for modern distributed, parallel, and reactive applications, representing a significant step forward in debugging technology.