Predicting Contextual Sequences via Submodular Function Maximization

Sequence optimization, where the items in a list are ordered to maximize some reward has many applications such as web advertisement placement, search, and control libraries in robotics. Previous work in sequence optimization produces a static ordering that does not take any features of the item or context of the problem into account. In this work, we propose a general approach to order the items within the sequence based on the context (e.g., perceptual information, environment description, and goals). We take a simple, efficient, reduction-based approach where the choice and order of the items is established by repeatedly learning simple classifiers or regressors for each “slot” in the sequence. Our approach leverages recent work on submodular function maximization to provide a formal regret reduction from submodular sequence optimization to simple cost-sensitive prediction. We apply our contextual sequence prediction algorithm to optimize control libraries and demonstrate results on two robotics problems: manipulator trajectory prediction and mobile robot path planning.

💡 Research Summary

The paper tackles the problem of sequence optimization in a contextual setting, where the ordering of items in a list must adapt to the current environment, goals, and perceptual information. Traditional sequence‑optimization methods produce a static ordering that ignores such context, limiting their applicability in domains like online advertising, search ranking, and robotic control libraries. The authors propose a general framework that leverages the properties of submodular functions to turn contextual sequence optimization into a series of simple cost‑sensitive prediction problems, one for each “slot” in the sequence.

Key technical contributions

1. Submodular reward modeling – The overall utility of a sequence is expressed as a submodular set function (f). Submodularity guarantees diminishing returns: the marginal gain of adding an item decreases as more items are already selected. This property enables a greedy algorithm to achieve a ((1-1/e)) approximation to the optimal sequence when the exact marginal gains are known.

2. Reduction to cost‑sensitive learning – Computing exact marginal gains online is intractable. The authors show that, for each slot (k), the marginal gain (\Delta_k(e)=f(S_{k-1}\cup{e})-f(S_{k-1})) can be treated as a cost (or negative reward) that a supervised learner can predict from the context (\mathbf{x}) and the current partial set (S_{k-1}). By training a cost‑sensitive classifier or regressor to estimate these gains, the greedy selection step becomes a simple inference call.

3. Regret reduction analysis – The paper provides a formal regret bound: if the learner’s average cost‑sensitive error at slot (k) is (\epsilon_k), then the expected utility of the learned sequence (\hat S) satisfies
   \