AVOCADO: The Streaming Process Mining Challenge

Streaming process mining deals with the real-time analysis of streaming data. Event streams require algorithms capable of processing data incrementally. To systematically address the complexities of this domain, we propose AVOCADO, a standardized challenge framework that provides clear structural divisions: separating the concept and instantiation layers of challenges in streaming process mining for algorithm evaluation. The AVOCADO evaluates algorithms on streaming-specific metrics like accuracy, Mean Absolute Error (MAE), Root Mean Square Error (RMSE), Processing Latency, and robustness. This initiative seeks to foster innovation and community-driven discussions to advance the field of streaming process mining. We present this framework as a foundation and invite the community to contribute to its evolution by suggesting new challenges, such as integrating metrics for system throughput and memory consumption, and expanding the scope to address real-world stream complexities like out-of-order event arrival.

💡 Research Summary

The paper introduces AVOCADO, a standardized challenge framework designed to evaluate streaming process mining (SPM) algorithms under realistic, real‑time conditions. Traditional process mining focuses on extracting models from static event logs, but modern enterprises increasingly generate continuous event streams that require incremental, low‑latency processing, handling concept drift, out‑of‑order events, and noisy data. Existing evaluation venues such as the Process Discovery Contest (PDC) are batch‑oriented and do not address these streaming-specific challenges.

AVOCADO addresses this gap by explicitly separating the concept layer (the abstract problem definition) from the instantiation layer (concrete algorithms, data generation, and evaluation procedures). The concept layer defines the core tasks for streaming SPM: continuous model generation, online conformance checking, and adaptive handling of concept drift. It also specifies a set of streaming‑specific performance metrics: accuracy, Mean Absolute Error (MAE), Root Mean Square Error (RMSE), processing latency, and robustness. The authors propose extending the metric suite in the future with system‑level measures such as throughput and memory consumption, thereby covering both predictive quality and resource efficiency.

To provide a reproducible benchmark, the authors design a synthetic data generation pipeline that creates a controlled drift scenario. Two random process models, p and k, are generated such that they share at least 85 % of activities but have completely different directly‑follows relations. Their union forms a “drift log” w. A streaming sequence is then constructed as p → w → k, where the first segment follows model p, the middle segment mixes events from both models (w), and the final segment follows model k. Each event in the validation stream is assigned a ground‑truth conformance label (1 for p, 0.5 for w, 0 for k). This design enables the evaluation of an algorithm’s ability to learn the initial behavior during a warm‑up phase, detect and adapt to drift, and produce accurate conformance scores throughout.

The evaluation procedure is fully automated. Participants must implement a BaseAlgorithm interface with two methods: learn(event) for the warm‑up phase and conformance(event) for online scoring. Submissions are required to run on CPU only, contain a start script without top‑level execution code, and be packaged for reproducible execution on the challenge platform. After the warm‑up stream is processed, the algorithm receives each event from the validation stream, returns a conformance prediction, and the platform computes the global error as the average distance between predictions and ground‑truth labels using the MAE/RMSE formulas. Processing latency is measured per event, and robustness is assessed by injecting noise and varying drift intensity.

The related‑work section surveys three families of streaming SPM approaches: (1) drift‑aware discovery methods, which often lack explicit noise handling; (2) declarative and low‑level event‑data techniques, which may sacrifice computational efficiency; and (3) online conformance checking methods, which provide real‑time deviation detection but rarely consider the full trade‑off between accuracy and resource consumption. AVOCADO’s contribution lies in integrating these aspects into a single, extensible benchmark, enabling systematic comparison across the entire pipeline rather than isolated components.

By publishing the source code, data generation scripts, and a public web interface, the authors invite the community to contribute new challenge scenarios, additional metrics, and real‑world stream datasets. The framework thus serves both as a reproducible baseline for academic research and as a practical testbed for industrial deployments that require real‑time process monitoring, anomaly detection, and adaptive process improvement. In summary, AVOCADO establishes a comprehensive, community‑driven platform for advancing streaming process mining, fostering innovation through standardized evaluation, and bridging the gap between theoretical algorithm development and operational applicability.