Unsolved

Outlier theory beyond non-degeneracy/invertibility assumptions (ReLU and zero diagonal entries)

Sourced from the work of Gerard Ben Arous, Reza Gheissari, Jiaoyang Huang, Aukosh Jagannath

§ Problem Statement

Setup

In the high-dimensional regime d,nd,n\to\infty with n/dα(0,)n/d\to\alpha\in(0,\infty), consider weighted sample-covariance/Hessian-type matrices of the form Hn=1n=1nDyyH_n=\frac1n\sum_{\ell=1}^n D_{\ell\ell}y_\ell y_\ell^\top arising from Gaussian-mixture features and data-dependent diagonal gates D=w(s)D_{\ell\ell}=w(s_\ell). Existing outlier analyses in this line of work typically assume non-degeneracy/invertibility-type conditions on effective diagonal weights.

Unsolved Problem

In the high-dimensional regime d,nd,n\to\infty with n/dα(0,)n/d\to\alpha\in(0,\infty), for weighted sample-covariance/Hessian-type matrices of the form Hn=1n=1nDyyH_n=\frac1n\sum_{\ell=1}^n D_{\ell\ell}y_\ell y_\ell^\top arising from Gaussian-mixture features and data-dependent diagonal gates D=w(s)D_{\ell\ell}=w(s_\ell), develop an outlier theory when P(D=0)>0\mathbb P(D_{\ell\ell}=0)>0: characterize limiting outlier locations and eigenvector alignments, and identify outlier phase transitions, without assuming diagonal weights are bounded away from zero.

§ Discussion

Loading discussion…

§ Significance & Implications

Extending the theory to ReLU-like degeneracy would broaden applicability of existing spectral results to commonly used gated models.

§ Known Partial Results

  • Arous et al. (2025): The source develops bulk/outlier results under non-degeneracy assumptions and indicates that the zero-mass (ReLU-type) degenerate case must be treated separately; no complete treatment of that degenerate extension is provided there.

§ References

[1]

Local geometry of high-dimensional mixture models: Effective spectral theory and dynamical transitions

Gerard Ben Arous, Reza Gheissari, Jiaoyang Huang, Aukosh Jagannath (2025)

Annals of Statistics (to appear)

📍 Section 1.5.1 (Multi-layer GMM classification); Remark 1.14.

Source paper where this problem appears. Metadata convention: `year` records the initial preprint year (2025), while this entry cites version `v3` dated January 22, 2026.

§ Tags