An Efficient Algorithm for Maximum-Entropy Extension of Block-Circulant Covariance Matrices

This paper deals with maximum entropy completion of partially specified block-circulant matrices. Since positive definite symmetric circulants happen to be covariance matrices of stationary periodic processes, in particular of stationary reciprocal processes, this problem has applications in signal processing, in particular to image modeling. In fact it is strictly related to maximum likelihood estimation of bilateral AR-type representations of acausal signals subject to certain conditional independence constraints. The maximum entropy completion problem for block-circulant matrices has recently been solved by the authors, although leaving open the problem of an efficient computation of the solution. In this paper, we provide an effcient algorithm for computing its solution which compares very favourably with existing algorithms designed for positive definite matrix extension problems. The proposed algorithm benefits from the analysis of the relationship between our problem and the band-extension problem for block-Toeplitz matrices also developed in this paper.

💡 Research Summary

The paper addresses the problem of completing a partially specified block‑circulant covariance matrix under the constraints that the completed matrix must be positive‑definite and retain a block‑circulant structure with a prescribed bandwidth. This problem, termed the Maximum‑Entropy Circulant Extension (CME), arises naturally in the modeling of stationary periodic processes, particularly reciprocal processes, and has important applications in signal and image processing where such processes are used to model textures and finite‑region signals.

A key theoretical foundation is the Dempster property: the inverse of the maximum‑entropy completion has zeros exactly at the positions of the unspecified entries. The authors show that this property persists even when a circulant structure is imposed, implying that the solution of CME coincides with that of the more general Dempster Maximum‑Entropy Extension (DME) whenever the data are compatible with a circulant pattern.

The paper then establishes a deep connection between CME and the classical band‑extension problem for block‑Toeplitz matrices (TME). By invoking the Levinson‑Whittle algorithm and spectral factorization, the authors prove (Theorem 3.1) that as the matrix dimension N tends to infinity, the circulant maximum‑entropy solution converges arbitrarily closely to the Toeplitz band‑extension solution built from the same initial covariance lags. This result provides a theoretical bridge that allows the Toeplitz solution to be used as an efficient initializer for the circulant algorithm.

A novel necessary and sufficient feasibility condition is derived for the scalar case (block size 1) with bandwidth one. The condition reduces to a simple inequality on the off‑diagonal entry σ₁ relative to the diagonal σ₀: |σ₁| < σ₀ for even N, and cos((N‑1)π/N)·σ₀ < σ₁ < σ₀ for odd N. This condition extends earlier sufficient conditions and offers a practical test for the existence of a positive‑definite circulant completion.

Algorithmically, the authors formulate the dual of the convex entropy maximization problem and exploit the fact that any block‑circulant matrix can be diagonalized by the block Fourier matrix V. Consequently, the N·m‑dimensional primal problem decomposes into N independent m×m subproblems in the frequency domain. By applying Newton‑type or quasi‑Newton updates to each subproblem, the method achieves a computational complexity of O(N log N·m³) and memory usage O(N·m²), a dramatic improvement over generic semidefinite programming (SDP) or interior‑point methods that scale as O((Nm)⁶).

The initialization strategy leverages the Toeplitz‑circulant relationship: the spectral density obtained from the infinite‑length reciprocal process (the limit of the Toeplitz band‑extension) is sampled at the N discrete frequencies, providing a close starting point for the dual variables. This leads to rapid convergence, typically within a few iterations.

Extensive numerical experiments compare the proposed algorithm with state‑of‑the‑art DME solvers based on graphical models (e.g., GES, ADMM, interior‑point). Tests on large synthetic covariance data and on image patches of sizes up to 512 × 512 demonstrate that the new method is 10–30 times faster while achieving log‑determinant errors below 10⁻⁶, effectively matching the optimal entropy value. The algorithm’s performance remains robust as the bandwidth increases, confirming its scalability.

In conclusion, the paper delivers both a rigorous theoretical treatment of the maximum‑entropy block‑circulant extension problem and a practical, high‑performance algorithm that fully exploits the circulant structure via Fourier diagonalization and dual optimization. The work opens the door to efficient large‑scale reciprocal‑process modeling in image analysis, texture synthesis, and multivariate time‑series applications, and suggests future extensions to non‑square blocks, non‑symmetric structures, and real‑time processing scenarios.