Distributed Linear Parameter Estimation: Asymptotically Efficient Adaptive Strategies

The paper considers the problem of distributed adaptive linear parameter estimation in multi-agent inference networks. Local sensing model information is only partially available at the agents and inter-agent communication is assumed to be unpredictable. The paper develops a generic mixed time-scale stochastic procedure consisting of simultaneous distributed learning and estimation, in which the agents adaptively assess their relative observation quality over time and fuse the innovations accordingly. Under rather weak assumptions on the statistical model and the inter-agent communication, it is shown that, by properly tuning the consensus potential with respect to the innovation potential, the asymptotic information rate loss incurred in the learning process may be made negligible. As such, it is shown that the agent estimates are asymptotically efficient, in that their asymptotic covariance coincides with that of a centralized estimator (the inverse of the centralized Fisher information rate for Gaussian systems) with perfect global model information and having access to all observations at all times. The proof techniques are mainly based on convergence arguments for non-Markovian mixed time scale stochastic approximation procedures. Several approximation results developed in the process are of independent interest.

💡 Research Summary

The paper addresses the problem of estimating a common unknown parameter vector θ∗ in a network of N agents, each of which observes a noisy linear function of θ∗ through a local sensing matrix Hₙ. Crucially, agents do not possess prior knowledge of either the global sensing model (the collection of all Hₙ) or the noise covariance matrices Rₙ associated with their observations. The authors propose an adaptive distributed linear estimator (ADLE) that simultaneously performs two coupled stochastic recursions: a state (estimate) update and a gain (weight) update.

The state update follows a consensus‑plus‑innovation structure:
xₙ(t+1) = xₙ(t) − βₜ ∑{l∈Ωₙ(t)}(xₙ(t) − x_l(t)) + αₜ Kₙ(t)(yₙ(t) − Hₙxₙ(t)).
Here, βₜ and αₜ are time‑varying step‑sizes that weight the consensus term (information exchange with neighbors Ωₙ(t)) and the innovation term (local measurement residual), respectively. The gain matrix Kₙ(t) is adaptively computed as
Kₙ(t) = (Gₙ(t)+γₜI_M)^{-1} Hₙᵀ (Qₙ(t)+γₜI{Mₙ})^{-1},
where Qₙ(t) is a running sample covariance of the local observations and Gₙ(t) is a consensus‑driven estimate of the global Fisher information matrix (the Gram matrix Σ_c = ∑_{n=1}^N HₙᵀRₙ^{-1}Hₙ). Both Qₙ(t) and Gₙ(t) are updated using all past data, making the overall procedure non‑Markovian.

The analysis rests on several mild assumptions: (i) global observability (Σ_c invertible); (ii) the communication graph Laplacians Lₜ are i.i.d. with a positive algebraic connectivity on average (λ₂(E