登录

公开问题

Differentiating through an eigendecomposition with near-degenerate eigenvalues

发布于 2026-09-02最近更新 2026-09-04318 次浏览

2

The standard backward rule for a symmetric eigendecomposition has 1/(λi−λj)1/(\lambda_i - \lambda_j) in it, which is fine right up until two eigenvalues approach each other. In our case they approach each other exactly where the interesting physics is, so the gradient goes to nonsense precisely where we need it.

Things I have tried: adding a damping ε\varepsilon to the denominator (biases the gradient in a way I cannot bound), and reformulating to avoid the decomposition (works for two of the four places it appears).

Is there a formulation whose derivative stays bounded through a degeneracy, given that the function being differentiated is smooth there even though the decomposition is not?

收藏这道问题
分享

提问者 · 2026-09-02Dan Okonkwo

1 个回答

2

The usual move is to stop differentiating through the decomposition and start differentiating through whatever invariant you actually want.

If the downstream quantity is a spectral function ∑ig(λi)viviT\sum_i g(\lambda_i) v_i v_i^T with gg smooth, the whole thing is a matrix function g(A)g(A) and its differential is the Daleckii–Krein formula — which has divided differences [g](λi,λj)[g](\lambda_i, \lambda_j) where your rule has 1/(λi−λj)1/(\lambda_i - \lambda_j). Those are perfectly well behaved at a degeneracy; they become g′(λ)g'(\lambda).

So the singularity was never in the problem, only in the intermediate you chose.

2026-09-04 · Mei Kurosawa