Nonasymptotic guarantees for bagged regularized M-estimators

Formal setup (matching the paper): under Assumptions A--D, let $X\in\mathbb R^{n\times p}$ have i.i.d. $N(0,1/p)$ entries, $y=X\theta+z$ with i.i.d. signal coordinates from $F_\theta$ and i.i.d. noise coordinates from $F_z$, and subsample index sets $I_m\subset[n]$ (with $|I_m|=k_m$) drawn independently as in Assumption B. For each $m\in[M]$, train the residual-loss regularized estimator

where each $(\mathrm{loss}_m,\mathrm{reg}_m)$ satisfies Assumption C, and Assumption D gives the additional moment/regularity conditions used in the risk-limit theory. Define the bagged estimator $\hat\theta_M:=M^{-1}\sum_{m=1}^M\hat\theta_m$ and the squared excess prediction risk

for $x_0\sim N(0,I_p/p)$ independent of training data. (When $\mathbb E[Z^2]<\infty$, full squared prediction risk adds the irreducible term $\mathbb E[Z^2]$.) Let $\mathrm{EST}$ be the source's data-dependent risk estimator, and let $R_M^{\infty}$ denote the deterministic proportional-limit risk for the corresponding proportional-asymptotic regime.

This setup follows Koriyama et al. (2025).