Unsolved

Small-SNR no-outlier regime and monotonicity in SNR

Sourced from the work of Gerard Ben Arous, Reza Gheissari, Jiaoyang Huang, Aukosh Jagannath

§ Problem Statement

Setup

Fix integers k,C1k,C\ge 1, mixture weights p1,,pk>0p_1,\dots,p_k>0 with b=1kpb=1\sum_{b=1}^k p_b=1, a class map c:[k][C]c:[k]\to [C], and an aspect ratio ϕ(0,)\phi\in(0,\infty). For each dd, let class means μ1,,μkRd\mu_1,\dots,\mu_k\in\mathbb R^d, collect μ=(μ1,,μk)Rd×k\mu=(\mu_1,\dots,\mu_k)\in\mathbb R^{d\times k}, and let parameters x=(x1,,xC)Rd×Cx=(x_1,\dots,x_C)\in\mathbb R^{d\times C}. For SNR parameter β>0\beta>0, data are generated by

BCat(p1,,pk),Y=μB+β1/2ξ,ξN(0,Id),B\sim\mathrm{Cat}(p_1,\dots,p_k),\qquad Y=\mu_B+\beta^{-1/2}\xi,\qquad \xi\sim N(0,I_d),

with observed class label c(B)[C]c(B)\in[C]. Define q:=C+kq:=C+k and the summary Gram matrix

G=(x,μ)(x,μ)Rq×q.G=(x,\mu)^\top(x,\mu)\in\mathbb R^{q\times q}.

Call GG feasible if G=(x,μ)(x,μ)G=(x,\mu)^\top(x,\mu) for some (x,μ)(x,\mu) (equivalently, G0G\succeq 0 and realizable in some ambient dimension).

Fix a class index α[C]\alpha\in[C]. Let smα(u):=exp(uα)/j=1Cexp(uj)\mathrm{sm}_\alpha(u):=\exp(u_\alpha)/\sum_{j=1}^C\exp(u_j) for uRCu\in\mathbb R^C, and for gN(0,Iq)g\sim N(0,I_q) define

fbH(β1/2g;G):=smα ⁣(G[C],b+G[C],[C]1/2β1/2g[C])(1smα ⁣(G[C],b+G[C],[C]1/2β1/2g[C])),f_b^H(\beta^{-1/2}g;G):=\mathrm{sm}_\alpha\!\big(G_{[C],b}+G_{[C],[C]}^{1/2}\beta^{-1/2}g_{[C]}\big)\Big(1-\mathrm{sm}_\alpha\!\big(G_{[C],b}+G_{[C],[C]}^{1/2}\beta^{-1/2}g_{[C]}\big)\Big), fbI(β1/2g;G):=(1{c(b)=α}smα ⁣(G[C],b+G[C],[C]1/2β1/2g[C]))2.f_b^I(\beta^{-1/2}g;G):=\Big(\mathbf 1_{\{c(b)=\alpha\}}-\mathrm{sm}_\alpha\!\big(G_{[C],b}+G_{[C],[C]}^{1/2}\beta^{-1/2}g_{[C]}\big)\Big)^2.

For M{H,I}M\in\{H,I\}, let νM(β,G)\nu_M(\beta,G) be the effective bulk law with Stieltjes transform SMS_M solving

1+zSM(z)=ϕb=1kpbE ⁣[SM(z)fbM(β1/2g;G)βϕ+SM(z)fbM(β1/2g;G)].1+zS_M(z)=\phi\sum_{b=1}^k p_b\,\mathbb E\!\left[\frac{S_M(z)\,f_b^M(\beta^{-1/2}g;G)}{\beta\phi+S_M(z)\,f_b^M(\beta^{-1/2}g;G)}\right].

Define FM(z,G)Rq×qF^M(z,G)\in\mathbb R^{q\times q} by

FijM(z,G):=βϕb=1kpbE ⁣[fbM(β1/2g;G)βϕ+SM(z)fbM(β1/2g;G)(giβ+(G)ib)(gjβ+(G)jb)].F^M_{ij}(z,G):=\beta\phi\sum_{b=1}^k p_b\,\mathbb E\!\left[\frac{f_b^M(\beta^{-1/2}g;G)}{\beta\phi+S_M(z)\,f_b^M(\beta^{-1/2}g;G)}\Big(\frac{g_i}{\sqrt\beta}+(\sqrt G)_{ib}\Big)\Big(\frac{g_j}{\sqrt\beta}+(\sqrt G)_{jb}\Big)\right].

Effective outliers are real roots of det(zIqFM(z,G))=0\det(zI_q-F^M(z,G))=0; define the right-outlier set

OM(β,G):={zR:det(zIqFM(z,G))=0, z>supsupp(νM(β,G))},M{H,I}.\mathcal O_M(\beta,G):=\{z\in\mathbb R:\det(zI_q-F^M(z,G))=0,\ z>\sup\operatorname{supp}(\nu_M(\beta,G))\},\qquad M\in\{H,I\}.

Unsolved Problem

For fixed feasible GG, characterize the dependence of right-outlier existence on β\beta. In particular, determine whether

β1β2  1{OH(β1,G)}1{OH(β2,G)},\beta_1\le \beta_2\ \Longrightarrow\ \mathbf 1\{\mathcal O_H(\beta_1,G)\neq\varnothing\}\le \mathbf 1\{\mathcal O_H(\beta_2,G)\neq\varnothing\},

and analogously with OI\mathcal O_I in place of OH\mathcal O_H. Also determine whether there exists β0(G)>0\beta_0(G)>0 such that

0<β<β0(G)  OH(β,G)=OI(β,G)=.0<\beta<\beta_0(G)\ \Longrightarrow\ \mathcal O_H(\beta,G)=\mathcal O_I(\beta,G)=\varnothing.

§ Discussion

Loading discussion…

§ Significance & Implications

Without this, the transition structure can be pathologically non-monotone, making global interpretation of SNR-dependent geometry unclear. A monotonicity/no-outlier-at-low-SNR theorem would complete the phase-transition narrative for this effective spectral framework. See Arous et al. (2025) for details.

§ Known Partial Results

  • Arous et al. (2025): The paper proves large-β\beta outlier existence but explicitly notes that a complementary small-β\beta no-outlier theorem is missing and that non-monotone scenarios are not ruled out. The problem is open as of January 22, 2026 (arXiv v3).

§ References

[1]

Local geometry of high-dimensional mixture models: Effective spectral theory and dynamical transitions

Gerard Ben Arous, Reza Gheissari, Jiaoyang Huang, Aukosh Jagannath (2025)

Annals of Statistics (to appear)

📍 Section 1.3.2 ("The effective spectrum at initialization and along the training trajectory"), immediately after Corollary 1.8 (comment on possible non-monotonicity and missing small-$\lambda$ no-outlier result), p. 12 (arXiv v3).

Source paper where this problem appears (first posted in 2025; this entry cites arXiv v3 from 2026).

§ Tags