Outlier theory beyond non-degeneracy/invertibility assumptions (ReLU and zero diagonal entries)
Sourced from the work of Gerard Ben Arous, Reza Gheissari, Jiaoyang Huang, Aukosh Jagannath
§ Problem Statement
Setup
In the high-dimensional regime with , consider weighted sample-covariance/Hessian-type matrices of the form arising from Gaussian-mixture features and data-dependent diagonal gates . Existing outlier analyses in this line of work typically assume non-degeneracy/invertibility-type conditions on effective diagonal weights.
Unsolved Problem
In the high-dimensional regime with , for weighted sample-covariance/Hessian-type matrices of the form arising from Gaussian-mixture features and data-dependent diagonal gates , develop an outlier theory when : characterize limiting outlier locations and eigenvector alignments, and identify outlier phase transitions, without assuming diagonal weights are bounded away from zero.
§ Discussion
§ Significance & Implications
Extending the theory to ReLU-like degeneracy would broaden applicability of existing spectral results to commonly used gated models.
§ Known Partial Results
Arous et al. (2025): The source develops bulk/outlier results under non-degeneracy assumptions and indicates that the zero-mass (ReLU-type) degenerate case must be treated separately; no complete treatment of that degenerate extension is provided there.
§ References
Gerard Ben Arous, Reza Gheissari, Jiaoyang Huang, Aukosh Jagannath (2025)
Annals of Statistics (to appear)
📍 Section 1.5.1 (Multi-layer GMM classification); Remark 1.14.
Source paper where this problem appears. Metadata convention: `year` records the initial preprint year (2025), while this entry cites version `v3` dated January 22, 2026.