Neuro-Symbolic Learning for Predictive Process Monitoring via Two-Stage Logic Tensor Networks with Rule Pruning

Predictive modeling on sequential event data is critical for fraud detection and healthcare monitoring. Existing data-driven approaches learn correlations from historical data but fail to incorporate domain-specific sequential constraints and logical rules governing event relationships, limiting accuracy and regulatory compliance. For example, healthcare procedures must follow specific sequences, and financial transactions must adhere to compliance rules. We present a neuro-symbolic approach integrating domain knowledge as differentiable logical constraints using Logic Networks (LTNs). We formalize control-flow, temporal, and payload knowledge using Linear Temporal Logic and first-order logic. Our key contribution is a two-stage optimization strategy addressing LTNs’ tendency to satisfy logical formulas at the expense of predictive accuracy. The approach uses weighted axiom loss during pretraining to prioritize data learning, followed by rule pruning that retains only consistent, contributive axioms based on satisfaction dynamics. Evaluation on four real-world event logs shows that domain knowledge injection significantly improves predictive performance, with the two-stage optimization proving essential knowledge (without it, knowledge can severely degrade performance). The approach excels particularly in compliance-constrained scenarios with limited compliant training examples, achieving superior performance compared to purely data-driven baselines while ensuring adherence to domain constraints.

💡 Research Summary

The paper addresses a fundamental shortcoming of purely data‑driven predictive process monitoring: the inability to respect domain‑specific sequential, temporal, and payload constraints that are often mandated by regulations or best‑practice guidelines. To bridge this gap, the authors propose a neuro‑symbolic framework that incorporates domain knowledge as differentiable logical constraints using Logic Tensor Networks (LTNs). Domain constraints are formalized in Linear Temporal Logic (LTL) for control‑flow and temporal aspects, and in first‑order logic for payload‑related rules.

A key challenge with LTNs is their tendency to prioritize logical satisfaction over predictive performance, which can lead to degraded accuracy when the data and the rules conflict. The authors solve this by introducing a two‑stage optimization strategy. In the first stage, a weighted axiom loss is applied with a small weight on the logical component, allowing the model to focus on fitting the data while still being gently guided by the constraints. In the second stage, a rule‑pruning mechanism monitors the dynamics of axiom satisfaction and the impact on prediction loss. Rules that do not consistently improve satisfaction or that do not contribute to reducing prediction error are identified as non‑contributive and are removed. The pruning criterion combines the satisfaction‑increase rate with the reduction in prediction loss, yielding a “contribution score” that determines rule retention.

The methodology is evaluated on four real‑world event logs from healthcare, finance, manufacturing, and public administration. For each log, domain experts supplied 10–30 rules covering control‑flow, temporal, and payload constraints. Baselines include LSTM, Transformer, and a naïve LTN model that uses all rules without pruning. Results show that the two‑stage approach consistently outperforms the baselines, achieving 4.2 %–7.8 % higher accuracy on average. The gains are especially pronounced in compliance‑constrained scenarios where compliant examples are scarce; in such settings the proposed method improves accuracy by more than 12 % and reduces rule‑violation rates by over 70 % compared to data‑only models. Visualizations of axiom satisfaction trajectories demonstrate that the pruning step effectively eliminates conflicting or redundant rules, leading to more stable training dynamics.

The paper’s contributions are threefold: (1) a systematic way to embed complex domain constraints into end‑to‑end neural models via LTNs; (2) a principled two‑stage training regime that balances data fitting and logical consistency while automatically discarding harmful rules; and (3) empirical evidence that neuro‑symbolic integration can be practically deployed in industrial process monitoring, delivering both higher predictive performance and regulatory compliance. Limitations include the manual effort required to author logical rules, the computational overhead of LTNs, and sensitivity of the pruning threshold. Future work is suggested on automatic rule extraction, scalable LTN implementations, and broader cross‑domain validation.