Representing and Reasoning with Qualitative Preferences for Compositional Systems

Many applications, e.g., Web service composition, complex system design, team formation, etc., rely on methods for identifying collections of objects or entities satisfying some functional requirement. Among the collections that satisfy the functional requirement, it is often necessary to identify one or more collections that are optimal with respect to user preferences over a set of attributes that describe the non-functional properties of the collection. We develop a formalism that lets users express the relative importance among attributes and qualitative preferences over the valuations of each attribute. We define a dominance relation that allows us to compare collections of objects in terms of preferences over attributes of the objects that make up the collection. We establish some key properties of the dominance relation. In particular, we show that the dominance relation is a strict partial order when the intra-attribute preference relations are strict partial orders and the relative importance preference relation is an interval order. We provide algorithms that use this dominance relation to identify the set of most preferred collections. We show that under certain conditions, the algorithms are guaranteed to return only (sound), all (complete), or at least one (weakly complete) of the most preferred collections. We present results of simulation experiments comparing the proposed algorithms with respect to (a) the quality of solutions (number of most preferred solutions) produced by the algorithms, and (b) their performance and efficiency. We also explore some interesting conjectures suggested by the results of our experiments that relate the properties of the user preferences, the dominance relation, and the algorithms.

💡 Research Summary

The paper addresses a fundamental problem that arises in many domains—selecting a collection of objects (e.g., services, components, team members) that not only satisfies a functional requirement but also optimizes non‑functional attributes according to user preferences. Existing approaches typically rely on numeric utility functions or total‑order importance models, which are often unrealistic because users express preferences qualitatively and may have only a partial ordering among attributes. To bridge this gap, the authors propose a two‑layer qualitative preference formalism.

First, for each attribute (a) (cost, reliability, latency, etc.) a strict partial order (\succ_a) over the attribute’s possible values is defined. This captures statements such as “low cost is preferred to medium cost” without assigning numerical scores. Second, a relative‑importance relation (R) over the set of attributes is introduced. Rather than forcing a total order, the authors require (R) to be an interval order, a well‑studied class that allows incomparabilities while still guaranteeing useful structural properties.

Combining these two layers yields a dominance relation (D) between two collections (X) and (Y). Roughly, (X) dominates (Y) if for every attribute (a) the value of (X) is at least as good as that of (Y) (according to (\succ_a)), and there exists at least one attribute that is strictly better and is not less important than any attribute where the two collections differ. The authors prove that when each (\succ_a) is a strict partial order and (R) is an interval order, the induced dominance relation (D) is itself a strict partial order (reflexive‑free, antisymmetric, transitive). This result is crucial because it guarantees the existence of a set of most‑preferred collections (the minimal elements of the dominance poset), analogous to a Pareto frontier but enriched by attribute importance.

On top of this theoretical foundation, three algorithms are designed to retrieve the most‑preferred collections:

1. Full‑Search enumerates all possible collections and filters them using (D). It is both sound (every returned collection is truly undominated) and complete (all undominated collections are returned) but suffers from exponential time and space complexity.

2. Priority‑Based Search exploits the interval‑order structure of (R). It processes attributes level by level, first eliminating dominated candidates with respect to the most important attributes, then refining the candidate set using less important attributes. The authors prove that, under the interval‑order assumption, this algorithm remains sound and complete while dramatically reducing the search space. Empirically it runs orders of magnitude faster than full‑search.

3. Sampling‑Based Search draws random or heuristic samples of collections and applies the dominance test. It guarantees weak completeness (at least one optimal collection will be found with probability approaching 1 as the number of samples grows) and offers the best scalability for very large problem instances, though it may miss some optimal solutions.

The experimental evaluation explores a wide parameter space: number of attributes (3–8), number of objects (5–20), density of intra‑attribute preferences, and the “tightness” of the importance relation (from near‑total order to highly partial interval order). Over 10,000 synthetic scenarios the authors compare the three algorithms on two metrics: (a) solution quality (how many of the true most‑preferred collections are recovered) and (b) performance (runtime and memory consumption).

Key findings include:

• Priority‑Based Search retrieves the full set of optimal collections in the vast majority of cases, with runtime reductions of 10–100× relative to full‑search. Its efficiency improves as the importance relation becomes more “layered” (i.e., clearer separation between importance levels).

• When the importance relation is almost a total order, the algorithm’s pruning becomes especially effective, essentially collapsing to a single‑level Pareto filter.

• Sampling‑Based Search scales to the largest instances (≥ 15 objects) with runtimes of only a few seconds, but its solution quality drops to 70–85 % of the optimum, especially when intra‑attribute preferences are dense.

• The experiments support a conjecture that the structural properties of user preferences (interval‑order vs total‑order) directly influence algorithmic tractability—a valuable insight for system designers who can encourage users to articulate preferences in a way that yields more efficient computation.

The paper concludes by outlining future research directions: incremental algorithms for dynamic preference updates, aggregation of multiple users’ qualitative preferences, and a prototype integration into a real‑time web‑service composition platform.

Overall, the contribution is twofold. Theoretically, it introduces a clean, mathematically sound model that captures both attribute‑level and cross‑attribute preferences without resorting to numeric utilities. Practically, it delivers algorithms that exploit the model’s structure to achieve sound, complete, or weakly complete retrieval of optimal collections, with empirical evidence of substantial performance gains. This work thus advances the state of the art in preference‑driven compositional reasoning and opens promising avenues for applying qualitative decision‑making in large‑scale, real‑world systems.