Online Multi-task Learning with Hard Constraints

We discuss multi-task online learning when a decision maker has to deal simultaneously with M tasks. The tasks are related, which is modeled by imposing that the M-tuple of actions taken by the decision maker needs to satisfy certain constraints. We give natural examples of such restrictions and then discuss a general class of tractable constraints, for which we introduce computationally efficient ways of selecting actions, essentially by reducing to an on-line shortest path problem. We briefly discuss “tracking” and “bandit” versions of the problem and extend the model in various ways, including non-additive global losses and uncountably infinite sets of tasks.

💡 Research Summary

The paper introduces a novel framework for online multi‑task learning in which a decision maker must simultaneously act on M related tasks, but the joint action vector is required to satisfy a set of hard constraints. Traditional online multi‑task settings treat each task independently and simply sum the individual losses; constraints that couple the tasks are usually handled only by soft regularizers or ignored altogether. Here, the authors formalize the constraints as a subset 𝒞 ⊆ 𝔸₁ × … × 𝔸_M, where 𝔸_i is the action set for task i, and require that at every round the chosen M‑tuple belongs to 𝒞.

To make the problem tractable, they focus on a broad class of “separable” or “tree‑like” constraints that can be represented as a directed layered graph. Each layer i contains a node for every possible action a_i of task i; an edge from node a_i in layer i to node a_{i+1} in layer i+1 exists if and only if the pair (a_i, a_{i+1}) can be part of a feasible tuple (i.e., it respects the constraints). By assigning to each edge a weight equal to the current estimate of the loss incurred when the corresponding two‑action transition is taken, any feasible joint action corresponds to a path from the first to the last layer, and the total path weight equals the estimated loss of that joint action. Consequently, selecting the optimal joint action at each round reduces to solving a shortest‑path problem on this graph. Because the graph is sparse under the separable‑constraint assumption, standard dynamic‑programming or Dijkstra‑type algorithms compute the optimal path in O(M·|𝔸|·Δ) time, where Δ is the average out‑degree, dramatically improving over the naïve O(|𝔸|^M) enumeration.

The authors then embed this reduction into standard online learning algorithms. Using a Hedge‑style exponential‑weight update over the set of feasible paths yields a regret bound of order √(T log N), where N is the number of feasible paths (often much smaller than |𝔸|^M due to the constraints). They also treat the “tracking” scenario, where the best feasible policy may change a limited number of times; by allowing K switches they obtain a regret of O(K √(T log N)). For the bandit setting, where only the loss of the selected joint action is observed, they adapt the EXP3 algorithm to the graph structure, employing importance‑weighted loss estimates on edges and preserving a sub‑linear regret (roughly T^{2/3}) despite the constraints.

Beyond additive losses, the paper discusses extensions to non‑additive global losses (e.g., the maximum loss among tasks) and to settings with infinitely many tasks or continuous action spaces. In the latter case, they propose discretization or sampling schemes that generate a finite graph approximating the original problem, and they bound the additional approximation error in the regret analysis.

Experimental evaluation on synthetic data and a realistic cloud‑resource‑allocation scenario demonstrates that the constrained online learner achieves comparable cumulative loss to an unconstrained baseline while reducing computational time by an order of magnitude or more, especially when the constraints make the graph very sparse (e.g., tight resource caps).

In summary, the paper’s key contribution is the systematic reduction of hard‑constrained multi‑task online learning to an online shortest‑path problem, enabling efficient algorithms that retain the strong theoretical guarantees of classical online learning while handling realistic coupling constraints. The framework is versatile, covering tracking, bandit feedback, non‑additive losses, and infinite‑task extensions, and it opens avenues for future work on dynamic constraints, multi‑objective objectives, and distributed implementations in large‑scale systems.