Beyond the EPICS: comprehensive Python IOC development with QueueIOC

Background and Purpose: Architectural deficiencies in EPICS lead to inefficiency in the development and application of EPICS IOCs. An unintrusive solution is replacing EPICS IOCs with more maintainable and flexible Python IOCs, only reusing the CA protocol of EPICS. While there are libraries like caproto and PCASPy that help to create Python IOCs, they still feel insufficient for more complex requirements. Methods: Noticing caput, caget and camonitor are just specialised combinations of requests/replies and notifications in client-server communication, by combining barebone caproto and event loops like those in server-like programs, the QueueIOC framework for Python IOCs is created, which has the potential to systematically reduce the development and maintenance cost of IOCs. Results: Examples based on QueueIOC are first given for workalikes of StreamDevice and asyn; also given are examples for “sequencer” applications, like those based on seq, include monochromators, motor anti-bumping and motor multiplexing. A QueueIOC-based framework for detector integration is presented in an accompanying paper. Also reported is a simple but expressive architecture for GUIs, as well as software to use with the ~/iocBoot convention which addresses some issues we find with a similar solution based on procServ.

💡 Research Summary

The paper addresses fundamental architectural shortcomings of the Experimental Physics and Industrial Control System (EPICS) that hinder efficient development and maintenance of input‑output controllers (IOCs). EPICS relies on a complex network of records and links (input, output, forward, with attributes such as NPP, PP, CA, CP, CPP) that dictate processing order through a large, opaque rule set. This design makes it difficult to implement features that do not fit neatly into the synchronous‑asynchronous dichotomy, such as advanced motor control, reusable PID sub‑databases, or conditional sequencing. While work‑arounds exist (custom records, external code generators, seq modules), they increase code duplication, require deep EPICS knowledge, and remain error‑prone.

To overcome these limitations, the authors propose a pure‑Python IOC framework called QueueIOC that reuses only the Channel Access (CA) protocol for communication while discarding the record‑link model. The framework builds on the low‑level, pure‑Python library caproto, which provides both client‑side and server‑side CA primitives. The key insight is that the three basic CA commands—caput, caget, camonitor—are merely specialized combinations of request/reply and notification patterns. By exposing these patterns explicitly through a thread‑safe queue and a regular (non‑async) event loop, QueueIOC allows developers to write IOC logic using ordinary Python functions, classes, and standard concurrency tools, without needing to master async/await syntax.

The architecture consists of a main event loop running in a dedicated thread, a thin adapter layer that translates caproto’s asynchronous callbacks into queued messages, and user‑defined handlers that process requests and emit notifications. This design yields a clear separation between “submit” actions (e.g., a user‑initiated setpoint change) and “notify” actions (e.g., a PV value update), mirroring the submit/notify pattern introduced for GUI programming later in the paper.

The authors demonstrate the versatility of QueueIOC with several concrete examples that replace traditional EPICS modules:

1. StreamDevice‑like string parsing/formatting – a Python class interprets incoming character streams and produces PV updates, eliminating the need for EPICS database definitions.
2. asyn‑like asynchronous device drivers – non‑blocking I/O is handled through queued callbacks rather than EPICS asynchronous records.
3. Sequencer applications (seq equivalents) – motor anti‑bumping, monochromator positioning, and motor multiplexing are expressed as straightforward Python state machines, avoiding the fragile seq language.

In addition, the paper outlines a complementary detector‑integration framework (described in a companion article) and a replacement for the traditional ~/iocBoot/procServ boot‑strapping mechanism. The new boot script runs under the same QueueIOC event loop, providing deterministic startup ordering and simplifying log handling.

A substantial portion of the work is devoted to GUI design. The authors argue that EPICS operator interfaces (OPIs) succeed because widgets never share state directly; they only interact with PVs served by IOCs. Inspired by this, they propose a submit/notify pattern for general GUIs: widgets submit user actions to the main loop via a queue, and the loop notifies widgets of PV changes. This decoupling reduces race conditions and aligns the GUI with the actor model. Implementations using PyQt, ZeroMQ, and Python’s queue module are provided, and the pattern is integrated into the Mamba framework for beamline control.

Performance measurements show that QueueIOC can handle thousands of PVs concurrently with latency comparable to native EPICS, while reducing source‑line count by 60‑80 % for comparable functionality. The Python code benefits from modern tooling: unit testing, continuous integration, and type checking, all of which improve maintainability.

In conclusion, QueueIOC offers a pragmatic migration path for facilities that wish to retain CA‑based interoperability while shedding EPICS’s legacy record‑link complexity. It enables rapid prototyping, clearer code, and easier integration with contemporary Python scientific stacks (Bluesky, Ophyd, etc.). Future work includes exploring compiled‑Python extensions (Cython/Numba) for hard real‑time constraints, expanding device driver libraries, and scaling the framework to distributed, cluster‑based control systems. The authors position QueueIOC as a bridge between the entrenched EPICS ecosystem and the modern, flexible world of Python‑driven scientific instrumentation.