Mining Treatment-Outcome Constructs from Sequential Software Engineering Data

Many investigations in empirical software engineering look at sequences of data resulting from development or management processes. In this paper, we propose an analytical approach called the Gandhi-Washington Method (GWM) to investigate the impact of recurring events in software projects. GWM takes an encoding of events and activities provided by a software analyst as input. It uses regular expressions to automatically condense and summarize information and infer treatments. Relating the treatments to the outcome through statistical tests, treatment-outcome constructs are automatically mined from the data. The output of GWM is a set of treatment-outcome constructs. Each treatment in the set of mined constructs is significantly different from the other treatments considering the impact on the outcome and/or is structurally different from other treatments considering the sequence of events. We describe GWM and classes of problems to which GWM can be applied. We demonstrate the applicability of this method for empirical studies on sequences of file editing, code ownership, and release cycle time.

💡 Research Summary

The paper introduces the Gandhi‑Washington Method (GWM), a semi‑automated analytical framework designed to discover treatment‑outcome constructs (TrOCs) from sequential software engineering data. GWM addresses a gap in empirical software engineering where researchers often have access to rich event logs (commits, reviews, builds, tests, releases, etc.) but lack a systematic way to compress these sequences, relate them to outcome variables (e.g., bug count, conflict count, cycle time), and automatically identify statistically significant patterns.

GWM consists of three distinct phases. In the Encoding phase, a domain expert maps each event type to a single character from a chosen alphabet. This step transforms a chronological trace of activities into a string, preserving order while enabling compact representation. The flexibility of the encoding allows analysts to tailor the granularity of the model to the research question, for instance encoding “commit”, “review”, and “test” as C, R, and T respectively.

The Abstraction phase takes the encoded strings and groups them into classes using regular expressions. By employing the Kleene star () and other regex operators, GWM abstracts away exact repetitions and focuses on the structural pattern of events. For example, both “CCCCRR” and “CR” are captured by the expression CR*. A depth‑first search over a hierarchy of regular expressions identifies parent‑child relationships, where a parent expression can generate all strings of its children. This hierarchy provides a compact view of the sequence space and reduces the dimensionality of the data dramatically.

In the Synthesis phase, each regular‑expression class is linked to the outcome measure of interest. The authors primarily use the non‑parametric Mann‑Whitney U test to compare the distributions of the outcome across different regex classes. A class is promoted to a treatment‑outcome construct when (1) the statistical test yields a p‑value below a predefined significance threshold, indicating a meaningful difference in the outcome, and (2) the regex represents a structurally distinct pattern relative to other significant classes. The resulting set of TrOCs constitutes the final output of GWM.

To demonstrate GWM’s applicability, the authors present three case studies.

1. File‑editing sequences: By encoding commit‑test‑review permutations, GWM discovers that the “C‑T‑R” (commit‑test‑review) pattern leads to significantly fewer bugs than alternative orders.
2. Code ownership: Encoding ownership changes as characters, the method identifies that long runs of contributions from a single developer (e.g., A*) correlate with lower maintenance effort.
3. Release cycle time: Verification periods are encoded as short (S), medium (M), and long (L). GWM finds that consecutive long periods (L*), alternating short‑long pairs ((SL)), and mixed patterns ((SML)*) each have a statistically distinct impact on the number of merge conflicts, with the identified patterns generally reducing conflict counts.

The authors argue that GWM offers several advantages: (i) the encoding step gives analysts expressive control over hypothesis formulation; (ii) regular‑expression abstraction dramatically compresses the search space while preserving essential temporal structure; (iii) automated statistical synthesis enables rapid, reproducible testing of many candidate patterns without manual enumeration.

Limitations are also acknowledged. The quality of the encoding depends heavily on expert judgment, introducing potential bias. The search for optimal regular expressions can become computationally intensive for large alphabets or very long sequences. Moreover, simultaneous or overlapping events and fine‑grained timing information are difficult to capture with a single‑character encoding, potentially limiting the method’s applicability to highly concurrent processes.

Future work suggested includes extending GWM to handle multiple outcome variables simultaneously, integrating Bayesian inference to model uncertainty, and building a fully automated pipeline that ingests raw repository logs, performs encoding, abstraction, and synthesis at scale for open‑source ecosystems.

In summary, the Gandhi‑Washington Method provides a novel, systematic approach for mining meaningful treatment‑outcome relationships from sequential software data. By coupling regular‑expression based abstraction with rigorous statistical testing, it enables researchers and practitioners to uncover actionable patterns that influence software quality, productivity, and risk, thereby bridging the gap between raw event logs and evidence‑based decision making in software engineering.