Differentiating the pseudo determinant

A class of derivatives is defined for the pseudo determinant $Det(A)$ of a Hermitian matrix $A$. This class is shown to be non-empty and to have a unique, canonical member $\mathbf{\nabla Det}(A)=Det(A)A^+$, where $A^+$ is the Moore-Penrose pseudo inverse. The classic identity for the gradient of the determinant is thus reproduced. Examples are provided, including the maximum likelihood problem for the rank-deficient covariance matrix of the degenerate multivariate Gaussian distribution.

💡 Research Summary

This paper addresses the fundamental problem of differentiating the pseudo-determinant of a Hermitian matrix, a function that arises in graph theory (e.g., Kirchhoff’s Matrix Tree Theorem) and statistics (e.g., the degenerate multivariate Gaussian distribution). The pseudo-determinant, defined as the product of a matrix’s non-zero eigenvalues, is a discontinuous function, particularly at points where the matrix rank changes. This discontinuity complicates the application of standard differential calculus.

The author’s primary contribution is the rigorous definition and characterization of a derivative for this function. The approach begins by expressing the pseudo-determinant as a limit of regular determinants: Det(A) = lim_{δ→0} det(A + δI) / δ^{n−k} for an n×n matrix A of rank k. To handle the discontinuity, the directional derivative is carefully defined only for directions B that share the exact same kernel (null space) as the point A. This leads to the definition of a class of derivative matrices ∇Det(A), characterized by the matrix equations A ∇Det(A) = A A⁺ Det(A) and ∇Det(A) A = A⁺ A Det(A), where A⁺ is the Moore-Penrose pseudo-inverse.

The central result of the paper is the identification of a unique, canonical member within this class. By leveraging known theorems—specifically, a Pythagorean theorem for the pseudo-determinant (relating Det²(A) to the sum of squares of all rank-k principal minors) and a representation theorem for the pseudo-inverse—the author proves that this canonical derivative is given by the elegant formula: ∇Det(A) = Det(A) A⁺. This canonical gradient has the same kernel as A itself. Furthermore, using matrix differential calculus, the total differential of the pseudo-determinant is shown to be d Det(A) = Det(A) tr(A⁺ dA), provided the differential dA also shares the kernel of A. This result elegantly generalizes the classic identity for the gradient of the regular determinant, to which it reduces when A is invertible (since then A⁺ = A⁻¹).

The power of this new differentiation rule is demonstrated through a key application in statistics: deriving the maximum likelihood estimator (MLE) for the singular covariance matrix Σ of a degenerate (rank-deficient) multivariate Gaussian distribution. The log-likelihood function involves both the log pseudo-determinant, log Det(Σ), and the quadratic form using the pseudo-inverse, tr(Σ⁺ R). Applying the new differentiation formulas alongside the known differential of the pseudo-inverse, the author computes the gradient of the log-likelihood. Setting it to zero yields the condition for the MLE, ^Σ. Under the key assumption that the kernel of the residual matrix R equals the kernel of Σ (meaning the data’s variation lies entirely within the model’s assumed subspace), the classical result is recovered: ^Σ = R/N, the sample covariance matrix. If this assumption does not hold, but the range of Σ is a predetermined subspace, the solution becomes the projection of R/N onto that subspace. This provides a complete generalization of the well-known full-rank Gaussian MLE to the degenerate case.

In summary, the paper successfully develops a consistent and practical calculus for the pseudo-determinant, resolving its discontinuous nature through a restricted directional derivative and identifying a simple, canonical derivative formula. The work bridges linear algebra and applied fields, offering essential tools for optimization problems in statistics and other disciplines where singular matrices are inherent.