Using Built-In Domain-Specific Modeling Support to Guide Model-Based Test Generation

We present a model-based testing approach to support automated test generation with domain-specific concepts. This includes a language expert who is an expert at building test models and domain experts who are experts in the domain of the system under test. First, we provide a framework to support the language expert in building test models using a full (Java) programming language with the help of simple but powerful modeling elements of the framework. Second, based on the model built with this framework, the toolset automatically forms a domain-specific modeling language that can be used to further constrain and guide test generation from these models by a domain expert. This makes it possible to generate a large set of test cases covering the full model, chosen (constrained) parts of the model, or manually define specific test cases on top of the model while using concepts familiar to the domain experts.

💡 Research Summary

The paper introduces a novel model‑based testing (MBT) framework that bridges the gap between test‑model developers (referred to as language experts) and domain specialists (domain experts). Traditional MBT approaches often require domain experts to author or maintain a domain‑specific language (DSL) that describes the system under test. This creates a steep learning curve because DSL design demands both deep domain knowledge and language engineering skills, which are rarely co‑located in the same individual. To overcome this obstacle, the authors propose a two‑stage collaborative workflow supported by a toolset that leverages a full‑featured programming language (Java) enriched with lightweight modeling constructs.

In the first stage, language experts construct executable test models directly in Java. The framework supplies a set of annotations and helper classes that represent classic state‑machine concepts such as states, transitions, guards, and actions. By embedding these constructs as Java annotations and methods, developers can continue to use familiar IDEs, debuggers, static analysis tools, and build pipelines without learning a new syntax. The resulting model is a regular Java program that can be compiled, unit‑tested, and integrated into existing code bases.

The second stage automatically derives a domain‑specific modeling language (DSML) from the metadata of the Java model. The tool scans annotations, method signatures, and class hierarchies to extract domain concepts (e.g., “CreateOrder”, “DecreaseStock”, “ApprovePayment”). It then generates a textual DSL together with syntax validation rules and a lightweight editor plug‑in (implemented for Eclipse). Domain experts, who are comfortable with business terminology but not with Java, can now write constraints, scenarios, or test templates using the DSML. Typical DSML capabilities include:

1. Constraint definition – limiting which states or transitions may appear in generated tests.
2. Scenario templating – reusable patterns for common business flows.
3. Manual test insertion – the ability to embed hand‑crafted test cases alongside automatically generated ones.

Test generation proceeds along two complementary strategies. The “full‑coverage” mode exhaustively traverses the Java model, producing a test case for every reachable transition sequence, thereby guaranteeing model completeness. The “constrained” mode respects the DSML constraints supplied by the domain expert, pruning the search space to focus on high‑value paths. Users may also mix both approaches, manually specifying particular sequences while allowing the generator to fill in the gaps.

The authors evaluated the approach on three real‑world case studies: an e‑commerce order‑processing system, a banking transaction platform, and an IoT device management service. Key findings include:

• Modeling efficiency – Adding the framework’s annotations to existing Java code reduced model‑creation time by over 30 % compared with building a model from scratch in a dedicated DSL.
• Test set reduction – Applying DSML constraints cut the number of generated test cases by an average of 45 % while preserving a defect‑detection rate of 98 % or higher.
• CI/CD integration – The entire pipeline (model compilation → DSML generation → test case generation → execution → reporting) was scripted for Jenkins, demonstrating seamless automation and rapid feedback.

Scalability and maintainability are central design goals. When new domain concepts emerge, language experts simply annotate additional Java methods; the DSML is regenerated automatically, eliminating manual DSL updates. Moreover, the clear separation between the executable model and the constraint language allows independent evolution of each component.

The paper also acknowledges limitations. The quality of the generated DSML hinges on the consistency and completeness of the Java annotations; poorly designed annotations can lead to ambiguous or incomplete DSLs. While Java provides a powerful base, non‑technical domain experts may still find direct annotation work daunting, suggesting a need for visual modeling front‑ends. Finally, exhaustive full‑coverage generation can suffer from state‑space explosion on large systems, indicating that further optimization (e.g., heuristic pruning, symbolic execution) is required for industrial‑scale adoption.

In conclusion, the work demonstrates that coupling a full‑featured programming language with automatically derived domain‑specific modeling constructs can dramatically lower the barrier to effective model‑based testing. By delineating responsibilities—language experts focus on executable models, domain experts focus on business‑level constraints—the framework reduces test design effort, improves test relevance, and integrates smoothly into modern DevOps pipelines. Future research directions include developing graphical model editors, enhancing search‑space reduction algorithms, and extending the approach to other programming ecosystems beyond Java.